Planet Linux Australia

I’ve just finished refactoring a large Go program, and the process was rather…. pleasant.

Static typing catches all those obscure errors I wouldn’t think about in a scripting language (Python, Perl, Ruby, etc). My process is:

• type :make in vim (I have a dummy Makefile in my Go project just for vim)
• vim jumps cursor to error (vim quickfix list)
• “oh, I shouldn’t do that” – fix (type type type)
• start again

Finish rather sooner than expected, run tests, smile in knowledge program is working properly.

Update

To quickly setup the make command for Go, type this in a Vim window:

:setlocal makeprg=go\ build\ \.

Or even better configure vim via your ~/.vimrc, for example:

autocmd BufRead *_test.go setlocal makeprg=go\ test\ \. autocmd BufRead *.go setlocal makeprg=go\ test\ \./..

Thanks Martin for the comment!

Update 8/Mar/13

Work on gsnmpgo has halted. Making the gsnmp C library multithreaded was proving too time consuming. Use http://github.com/soniah/gosnmp instead.

Previously…

I recently released gsnmpgo – a Go/CGo snmp library using gsnmp. Pull requests welcome!

From the gsnmpgo documentation, here’s an example of usage:

// do an snmp get; RFC 4088 is used for uris uri := `snmp://public@192.168.1.10//(1.3.6.1.2.1.1.1.0)` params := gsnmpgo.NewDefaultParams(uri) results, err := gsnmpgo.Query(params) if err != nil { fmt.Println(err) os.Exit(1) } // check your results gsnmpgo.Dump(results) // turn on debugging gsnmpgo.Debug = true

The guys training with wooden swords (Bokken) and metal swords.

http://www.youtube.com/watch?v=xETBPYwqXMI

(Just a summary of Stack Overflow “How do you Remove an Invalid Remote Branch Reference from Git?”).

To delete a local tracking branch (without deleting the remote branch), do:

git branch -rd remote/branch

And of course to delete the remote branch:

git push remote :branch

Occasionally a gc will help, but usually shouldn’t be used:

git gc --prune=now

So, when would you want to use this? Let’s say the repository you’re tracking has a lot of branches (eg the Linux Kernel). You start tracking branch “foo”, do some work with it, merge some of it in to your branch “bar”, then push “bar” up to the remote repository. Or, you’ve got a whole lot of dev branches you’ve merged pushed to your backup repository and also merged into your master branch.

In either case you’ve got a collection of tracking branches you don’t want to see anymore, so clean them up:

% git branch -r soniah/dev.a soniah/dev.b soniah/dev.c soniah/dev.d soniah/master % git branch -rd soniah/dev.a Deleted remote branch soniah/dev.a (was deadbeef). % git branch -rd soniah/dev.b Deleted remote branch soniah/dev.b (was deadbeef). # now remote branches is cleaner: % git branch -r soniah/dev.c soniah/dev.d soniah/master

Today I released soniah/gosnmp – an update of alouca/gosnmp.

Many, many thanks to Andreas Louca for writing alouca/gosnmp. The major difference between his version and soniah/gosnmp is that the latter has tests written. (However the code could do with refactoring). The tests were used to find and correct errors in the following SNMP BER Types:

• Counter32
• Gauge32
• Counter64
• OctetString
• ObjectIdentifier
• IpAddress

Also, this version contains functions for treating the returned snmp values as *big.Int (convenient, as SNMP can return int32, uint32, and uint64 values)

 

Git has a “force push” option (git push -f remote branch), but it doesn’t have a “force pull” option (like git pull -f remote branch).

This works:

% git fetch remote branch % git reset --hard FETCH_HEAD % git clean -df

Or, as a function for your bash/zsh config file:

gpuf () { # git pull -f $1 remote=${1:?"need remote to force pull from"} current_branch=$(git symbolic-ref -q HEAD) current_branch=${current_branch##refs/heads/} current_branch=${current_branch:-HEAD} if [ $current_branch = 'HEAD' ] ; then echo echo "On a detached head. Exiting..." exit 1 fi git fetch $remote $current_branch git reset --hard FETCH_HEAD git clean -df }

I’ve recently been working on a large Go project, and one of the deliverables was that the project be buildable using Jenkins. I was unfamiliar with Jenkins, and there didn’t seem to be any documentation around on how to build Go executables.

Project Structure

First of all an aside on project structure. For my first project I had a GOPATH of ~/go and the following directory structure:

~/go |--bin |--pkg |--src

But as I wrote more Go projects, it made more sense to separate each project into it’s own directory structure:

~/go |--project1 |--bin |--pkg |--src |--project2 |--bin |--pkg |--src

With this directory structure I set GOPATH on the command line or in a Makefile:

% cd ~go/project2/src/github.com/soniah/gosnmp % GOPATH=~go/project2 go build Makefile

For smaller projects you can just produce a binary using go run or go build. But a Makefile will be needed for larger projects, as they will have other deliverables besides a binary – for example manpages or an operating system installer like a .deb for Ubuntu/Debian.

GOROOT := /usr/lib/go GOPATH := /var/lib/jenkins/workspace/go/project2 myhostname := $(shell hostname) ifeq (${myhostname}, laptop) GOPATH := /home/sonia/go/project2 else ifeq (${myhostname}, testmachine) GOPATH := /home/u1234/go/project2 GOROOT := /usr/local/go endif build: build-stamp build-stamp: file1.go file2.go file3.go # always format code GOPATH=$(GOPATH) go fmt $^ # binary GOPATH=$(GOPATH) go build -o project2 -v $^ # docs markdown README.mkd > README.html help2man --no-info --include=help2man.roff --name "Project2" ./project2 > project2.roff man -Tps -l project2.roff > project2-man.ps ps2pdf project2-man.ps project2-man.pdf # mark as done touch $@ Jenkins

With a working Makefile, building under Jenkins will now be easier. The objective is to have Jenkins automatically build a new binary/package whenever a developer pushes to one of the git/mercurial/bzr repos that makeup the various components of your project.

However if your project contains multiple components, you’ll soon come across a problem. For example Project2 was using github.com/droundy/goopt and github.com/mattn/go-sqlite3. To see the problem, create a New Job using Build a free-style software project. Under Source Code Management, choose something like Git. Under the second Advanced button, you’ll need to change the option Local subdirectory for repo (optional) to point to the path of a component’s repo. But this setting is a global setting for all git repos – so the build won’t work as you add a second and third component.

The solution is to install to Jenkins the Multiple SCMs Plugin. Then in your Jenkins Job you’ll be able to set the local subdirectory for each component. For example in Project2:

• the goopt local subdirectory was set to project2/src/github.com/droundy/goopt
• the go-sqlite3 local subdirectory was set to project2/src/github.com/mattn/go-sqlite3

Other Jenkins  Settings

Here are some other useful setting for building Go projects on Jenkins:

• if you’re building 32 and 64 bit binaries (on different build servers), use the setting Restrict where this project can be run with something like “32bit&&precise&&ubuntu”
• separate out your Go code from other languages – Advanced Project Options, Use custom workspace, “/var/lib/jenkins/workspace/go”

There is already a great article on Profiling Go Programs. However that article only discusses how to profile a standalone binary – what about a library?

For example, I’ve been working on the GoSNMP SNMP library, here’s how I profiled it (it wasn’t obvious):

# produce cpu profiling information from the tests - this part was well documented % go test -cpuprofile cpu.out # compile the test binary to pkg.test but do not run it (from `go help test`) # this part wasn't obvious % go test -c # now run pprof using `go teste -c` output # using gosnmp.test - this part wasn't obvious % go tool pprof gosnmp.test cpu.out

Doing a memory profile was similar:

% go test -memprofile mem.out % go test -c % go tool pprof gosnmp.test mem.out

There are many pages out there discussing how to recover an Ubuntu encrypted home directory (see also below). These are merely notes for my future reference; they need tidying at there may be errors/mis-attributions in it.

Start by booting from an Ubuntu Live CD.

Passwords

Three different “passwords” are referred to when recovering:

* boot password ie the password used when your laptop is first booted and the partitions are decrypted

* user password ie your unix account password

* mount password – will look something like f0bddb4c533fddb2c89e890098ed65d1. The one that you didn’t write down when prompted to do so… See “Recovering Your Mount Passphrase” https://help.ubuntu.com/community/EncryptedPrivateDirectory#Recovering_Your_Mount_Passphrase

Partitions and Disk Layout

If you selected the default Ubuntu encryption setup, the partitions will be laid out like this:

/dev/sda1 * 2048 499711 248832 83 Linux /dev/sda2 501758 976771071 488134657 5 Extended /dev/sda5 501760 976771071 488134656 83 Linux

• /dev/sda1 contains /boot ie kernel and grub
• /dev/sda5 is an encrypted partition (crypto_LUKS) that contains LVM. The Logical Volumes will be for /root, /home and swap. /home will be encrypted with a second level of encryption if you chose “encrypt home directory” during installation.

Mounting Encrypted LVM Partition

* confirm /dev/sda5 is the correct partition [1]:

cryptsetup -v luksDump /dev/sda5

* mount the encrypted partition containing the LVM volumes:

cryptsetup -v luksOpen /dev/sda5 sda5_crypt vgdisplay (you may need to rename the volume group using vgchange if it conflicts with an existing one. A good motivation for using different VG names on each machine) lvdisplay | less mkdir /mnt/home mount -t ext4 /dev/vg/home /mnt/home Mounting Encrypted Home, LUKS vs eCryptfs

The partition was encrypted with LUKS, and /home will be encrypted with a second level of encryption (eCryptfs) if you chose “encrypt home directory” during installation.

Note for future installs: you’d only want to have both if multiple people were using the same machine. Otherwise use only LUKS especially for laptops; eCryptfs is an extra hurdle during recovery and doesn’t give extra security. Also, using only LUKS is more secure than only encrypting your home directory using eCryptfs:

* it will encrypt other things beside /home eg swap, /tmp

* you’ll only type in your long LUKS passphrase occasionally (ie at reboot), whereas the eCryptfs password will be typed in every time you login or unlock the screen ie will be more vulnerable to shoulder-surfing, and more likely to be too short

However, eCryptfs does have some advantages (http://www.privacydusk.com/tag/ecryptfs-vs-luks/):

* All the cryptographic metadata is stored in the header of the file. This means that the encrypted file can be copied and moved from one location to another not leaving any metadata behind

* Files can be encrypted with multiple keys so that multiple different users can have access to encrypted but shared files. You can have different files encrypted by different users and each user can access only his files

Automated Approach [2]

* “remount” /mnt/home on home:

umount /mnt/home

mount -t ext4 /dev/vg/home /home

# add a user with the same name as the broken system

adduser –no-create-home sonia

su sonia

ecryptfs-mount-private

Manual Approach [3]

The Ubuntu documentation on EncryptedPrivateDirectory has lots of information [4]. These commands are copied from there, in case the page moves or disappears.

(((

sudo ecryptfs-add-passphrase –fnek

Passphrase: (Enter the mount passphrase you recorded when you setup the mount–this passphrase is different from your login passphrase.)

You should now get two lines looking like this:

Inserted auth tok with sig [9986ad986f986af7] into the user session keyring

Inserted auth tok with sig [76a9f69af69a86fa] into the user session keyring (write down the second value in the square brackets)

mkdir /mnt/Private

mount -t ecryptfs /mnt/home/sonia/.Private /mnt/Private

Selection: 3 (use a passphrase key type)

Passphrase: (Enter the mount passphrase you recorded when you setup the mount–this passphrase is different from your login passphrase.)

Selection: aes (use the aes cipher)

Selection: 16 (use a 16 byte key)

Enable plaintext passthrough: n

Enable filename encryption: y (This and the following options only apply if you are using filename encryption)

Filename Encryption Key (FNEK) Signature: (the value you wrote down from the second line above)

)))

See Also

[1] http://blog.miketoscano.com/?p=72

[2] http://goshawknest.wordpress.com/2010/04/16/how-to-recover-crypted-home-directory-in-ubuntu/

[3] https://help.ubuntu.com/community/EncryptedPrivateDirectory

[4] https://help.ubuntu.com/community/EncryptedPrivateDirectory#Recovering_Your_Data_Manually

Git bisect is a great tool for finding bugs in a program. But many examples show manual uses of git bisect – here’s an example of automating the process using git bisect run.

tl;dr

Using git bisect run is easy if you’ve make small atomic commits and you have good tests. run makes a large debug easier (compared to manually doing git bisect good and git bisect bad) – you’re less likely to make errors due to boredom. And run means you can use an iterative process – use rebase to split bad commits then just run again.

Example

So I had an elusive bug in a long running process (an snmp poller, calculator and aggregator for a large network). I had a point where the program was good, but I’d added more features since good and now results were bad. The first step was to write a shell script to be called from git bisect run:

% cat bisect.sh #!/bin/bash # copy this to ~ before running with `git bisect run ~/bisect.sh` cp ~/Makefile . make clean # make modifies manpage output, so stash after build if make &> /dev/null ; then git stash git stash clear else git stash git stash clear exit 125 fi sudo cat /var/tmp/empty > /var/log/abc/abc-poller.log sudo ./abc-poller --tmp --once -d 2 -c 150 || exit 125 echo "=== poller finished" percent=`godir=/var/tmp/data/abcmon/poll_queue/new ~/checker | \ tail -1 | awk '{print $5}' | awk -F. '{print $1}'` echo "=== percent is $percent" (( percent < 5 ))

Things that make writing the test script easier:

• first get it working outside of git bisect run – usually means echoing results along the way
• already having a test suite that produces a quantified pass/fail output, In my case the I had already written the checker program, whose last line of output contained a “percentage failure” figure

Next step was having an abbreviated log of commits to refer to:

% git log --oneline f00f232 sql.go - better debugging # bad 9780d44 dummy .gitignore, so out dir preserved 0de0796 Makefile for nsch1abcs01 a2f6c96 defaults - 20 workers, udp 15 b8ee3d9 GOMAXPROCS() 04f21ba start v0.0.2 dbc6a60 Makefile: Jenkins as default for env vars 557a5e3 more work on stats ef6a453 remove excessive debugging ccc4644 remove file buffering - wasn't writing..??? 98bf4b1 stats write failing 9a9682d move type queue_t struct 467aeed buffered writes for queue file 148a8cc stats: + device_run, device_ok c61be42 done chan *Stats_t; calculate_value() bool 0e41461 debugging - print out device_id as %5s 7cbe167 default workers 5000, correct stop/start commands 544e8d7 gather statistics 6990cf2 rename data chan to device_id 5e8562b rename sql -> sqlconn; global var fd52d89 remove dead code 4d7b9b1 device_for() - err if count != 1 6ceb305 deb: fail if version main.go isn't same changelog 5ce1855 deb: rules producing abc-poller_0.0.1_amd64.deb db5bff4 deb: cleaned up Makefile, roffs 76674c2 JSON -> SQL; version 0.0.1 4cf198e deb: basic removal of 64 references d980f26 deb: rename 64 to vanilla b933570 deb: remove 32 bit stuff 970bd82 current debug level is 2; adjust output 70f8921 Revert "deb build - don't init, cron while testing" 855ce00 default debug is 2; misc tidy f3d80e1 remove timeoutOpt - no longer used 48ced38 misc tidys before release a3fe13c runonceOpt, revert cycling code # good 24cd998 use passed in udpOpt 5ca1830 remove stash/sender.old.go 06b9566 remove gsnmpgo; use gosnmp bffd430 rules: add note about "too many open files"

Mark bad and good, start the run, go and have a coffee :-)

% git bisect start f00f232 a3fe13c % git bisect run ~/bisect.sh # lots of output

I get the result that the ominously named 557a5e3 more work on stats is the first bad commit – I remember it as one of those large “kitchen sink” commits done at the end of the day. So “first rule of fightclub git” remembered – always do small atomic commits.

I have a useful shell function gri() – I used that to interactively rebase and break up 557a5e3 into many small commits:

gri () { git rebase -i HEAD~${1:-7} }

After rebasing git log looked like this – notice the many small commits named “bisect1″ etc:

382b3ee defaults - 20 workers, udp 15 11db314 GOMAXPROCS() 4b547ea start v0.0.2 c92feef Makefile: Jenkins as default for env vars 8fcc595 bisect6: calc/noncalc ad3a18f bisect5: tweak debug msgs 311bb8d bisect4: mv stats init 957c234 bisect3: remove stats from send_gosnmp() 3c01d62 bisect2: use Add(); Calcs/NonCalcs 947cb27 bisect1: move Stats_t; Add() ef6a453 remove excessive debugging ccc4644 remove file buffering - wasn't writing..??? 98bf4b1 stats write failing

And here’s the real win of writing bisect.sh – you can just keep rebasing and running until you’ve narrowed down the bad code to a few lines:

=== poller finished === percent is 49 947cb27fd57642dc545ee23090d7ae8fd8b14b3f is the first bad commit commit 947cb27fd57642dc545ee23090d7ae8fd8b14b3f Author: Sonia Hamilton <sonia@snowfrog.net> Date: Thu Mar 14 10:02:42 2013 +1100 bisect1: move Stats_t; Add()

I do another interactive rebase, fix the logic error, and then HEAD is good.

One thing people might be surprised about my job is that although I'm in network operations I write a lot of code, mostly for tools we use to monitor and maintain the network, but I've also had a few features and bug-fixes get into Google-wide (well, eng-wide) internal tools. This has lead me to use a bunch of languages, quite probably more than most Google software engineers.



Languages I've used in the last three months:

• C++
• Java
• Go
• Python
• bash
• Javascript & HTML (Including several templating languages)

In the two years since I started there's also:

• perl
• tcsh
• PHP
• SLAX (An alernate syntax version of XSLT)

These end up being a fairly unsurprising mix of standard sysadmin, web and systems programmer faire, with the real outliers being Go, the new c-ish systems language created at Google (several of the people working on the language sit just on the other side of a wall from me), and tcsh & SLAX which come from working with Juniper's JunOS which is built on FreeBSD with an XML-based configuration.

When the Syrian Electronic Army hacked The Onion’s twitter account, what did they do to cause panic and mayhem? Post real news stories?

Yesterday I ran in the City2South. It was a beautiful day for it, and I really enjoyed the run. Thanks to generous support of my donors, I raised $252 for the Heart Foundation.

My official time was 1:20:41, which I'm really happy with. I'd only ever run 14km on the preceding Tuesday, and I ran that in 1:32:57, so to do this run 12 minutes faster, on a completely different course felt like quite the accomplishment. I also ran personal best times for 5K and 10K. It's hard to believe that the guy who came first ran it in 44 minutes.

the course was really nice, except for running up Highgate Hill at kilometre 12. That was a bit harsh, but I managed to run all the way up it, nonetheless. It only hit me this afternoon when I was replaying the course in my mind just how far I ran. My normal 10K course doesn't feel all that long because it loops back on itself a lot, so it's deceptive how much ground I cover.

From a technical perspective, the race was done very well. I liked that they had a Facebook app that in real time posted updates when I crossed the start, 5K, 10K and finish lines, and the official results were online by the time I got home. That said, as I write, the website is throwing all sorts of errors when I try to download my official finishing certificate, or see my photos and finish line video.

All things considered, it was a pretty nice way to spend a Sunday morning. I was up at 5am to be on the 5:44am ferry from Hawthorne, and back home again by 10am.

I'm very keen to try running a half marathon now, but my next run is the 10km Bridge to Brisbane in September. That one will be more interesting because presumably it involves running up the Gateway Bridge, and I'll be pushing Zoe in a jogging stroller. I'm not expecting any personal best times for that one.

I just want to make sure that everyone knows that the LCA2014 call for proposals is open. There are two calls this time around -- a call for proposals and a call for miniconfs. The call for proposals closes on 6 July, so you don't have heaps of time left to submit something.



So, if you're interested in speaking at linux.conf.au 2014, in Perth between 6 and 10 January 2014 you should hit up those CFPs now!



Tags for this post: conference lca2014 cfp

Related posts: LCA 2006: CFP closes today; Got Something to Say? The LCA 2013 CFP Opens Soon!; Call for papers opens soon



Comment

Last Saturday I had my first Summits on The Air (SOTA) attempt on top of Mount Remarkable here in South Australia.

As a first step on Friday I registered my SOTA attempt on the Sotawatch web site

On Saturday morning I started by testing my FT-817 and Alexloop magnetic loop antenna at our camp. While tuning up I managed to talk to a VK2 (portable in VK5) who was few 100 km away in the Flinders ranges. Good test.

I then hiked for a few hours to get to the top of Mt Remarkable, set up my radio and antenna, and called CQ on 40m and 20m. Alas, I made no contacts. However it was so nice to experience S0 noise on 40 and 20m, so much different to my urban S9 hash experience on those bands. I couldn’t hear much activity on 40m but could hear many international stations on 20m. They just couldn’t hear me!

The members of the SOTA Australia Yahoo Group have been most helpful with many suggestions on how I can do better next time. In particular I can “self spot” using a smart phone app like sotagoat or a web site. I’ll certainly give it another go in future.

Some pictures of my little adventure:

In the last picture the magnetic loop is just behind my head, the FT-817 just visible above the white note book. I use a 1m dowel as the antenna mast which doubles as a walking stick for the hike. A large part of this walk is over paths covered with large rubble. as shown in the pictures above. I was told this is from ancient volvanic activity. The rubble moves a bit under your feet, making for slow going. There is a light plane crash about 2/3 of the way up – the alloy remains of the plane still shiny after 30 years. The walk was about 6 hours return for me from the caravan park at the base of Mt Remarkable. However I am a slow walker, and had a sore knee from a bike crash a few days before!

Most people will have seen the Call Me Maybe blog posts about data loss in the face of network partition. A few times in that series the author discusses “CRDT” as an alternative approach to the concurrency problem.

Midway through the last post in the series is what is almost an off-the-cuff comment, but I think it’s everything:

“Consistency is a property of your data, not of your nodes.”

We tend to get overwhelmed with replication configurations, high-availability solutions, sharding strategies, and worrying about how a given database will react under various failure modes.

And yet, the essential truth that we're so busy worrying about what's stored on disk that we can forget that don't care about consistency of what's on disk. We need to care about the consistency of our data. It's easy for a misbehaving program to write garbage, but not to worry! we're absolutely certain that garbage is consistently replicated across the cluster. Yeah, well done there.

So the much bigger challenge in high-availability distributed systems, is making sure we have sane rules for propagating changes so that we can have a safe view of our data.

About 10 years ago I was working with a Java based object-oriented database (which is a grandiose name for what was as much a disk-backed datastore as anything else, but if you're morbidly curious about what sort of API such a beast would have, you can read about db4o in a series of posts I wrote about it). It was surprisingly easy to use, and came along at a time when I was prepared to do just about anything to escape the object-relational mapping hell.

They got significant adoption in embedded devices where zero-administration is a necessity and developers don't wanting to deal with the machinery of a full scale RDMBS just to store e.g. configuration parameters. But surprise, it wasn't long before users started asking for replication features. Now, usually when you hear that term you think of master/slave replication being done at database engine level in a high-availability setup. In this case, however, they had disconnected devices re-establishing connectivity to enterprise datastores, and because of that you had to cope with significant conflicts when it came time to synchronize.

Because the data model was articulated in terms of Java code (to a naive first approximation, you were just storing Java objects), you had the data model living in the same place as the application code, domain layer, and validation logic. This meant that when it came time to cope with those conflicts, the natural place to put do that was in the same Java code. This was interesting, because for just about every other database engine out there data is opaque. Oh, sure, RDBMS have types (though that there are people who think VARCHAR(256) actually tells you anything useful remains a source of wonder; alas, I digress), but if you have a high availability configuration and you've allowed concurrent activity during a network partition, then you have to deal with diverged replicas and thus have to merge them. Database doesn't know what to do; how could it? No: consistency is a property of your data, not the datastore; the rules to decide how to synchronize are a business decision, so where better to put it than in the business logic?

Peter Miller suggests the example of booking flights: multiple passengers can end up allocated the same seat on an oversold flight, but the decision about who gets which seat happens at check-in and conflict resolution is a business one made by the airline staff, not the database.

Throughout the Jepsen posts, you'll see occasional mention of "CRDTs" as an alternative to the problems of attempting to achieve simultaneous write safety in a distributed system. Finding out just what a CRDT is took a bit more doing that I would have expected; hence wanting to write this post.

Convergent and Commutative Replicated Data Types

It's easy to have Consistency when you impose synchronous access to your data. But the locks needed to give that property don't scale to distributed systems; you need to have data that can cope with delay. The idea of self-healing systems have been around for a while, but there hasn't been much formal study of what data types meet these requirements. If you're at all interested, I'd encourage you to have a read of "A comprehensive study of Convergent and Commutative Replicated Data Types" by Shapiro, Preguiça, Baquero, and Zawirski.

http://hal.inria.fr/docs/00/55/55/88/PDF/techreport.pdf

They use set notation and a form of psuedocode to describe the different data types which all makes the read a bit more serious than it needs to be, but having had my head buried in this paper for a few days I can say the effort has paid off. They articulate a set of conditions that would make either a state based system able to handle merges — which basically works out because the requirement is for the datatype to be a join semilatice; if it is, then they show the replicas will converge — or an operation based one (aka command pattern to us programmer types) — where the requirement is for manipulations of the datatype to be commutative, and if so, ditto [They also show these are equivalent, which is handy].

Here's an schematic illustration of a state-based convergent replicated data type:

The idea being that if you have a merge function, then it doesn't matter where a state change is made; it will eventually make its way to all replicas.

Which raises the topic of eventual consistency. Anyone who has worked with Amazon S3 has discovered (the hard way, inevitably) that mutating an existing value has wildly undefined behaviour as to when other readers will see that change. CRDTs, on the other hand, exhibit "strong eventual consistency" (or perhaps better "strong eventual convergence", as Murat Demirbas put his analysis of the topic), whereby the propagation behaviour is well defined.

At first you'd think that this would seriously cramp your style, but the real contribution of the paper is they then explore around a bit and examine a number of different datatypes that meet these requirements.

The paper also includes an impressive reference list & discussion of prior art in the space, so it's worth a read. There's also "Conflict-free Replicated Data Types" by the same authors which formalizes SEC http://pagesperso-systeme.lip6.fr/Marc.Shapiro/papers/CRDTs_SSS-2011.pdf

Back to the effect of network partitions on data safety:

What about Ceph?

Good question.

What I would be interested in now is how Ceph's various inter-related pieces hold up in the face of the sort of aggressive network partition testing conducted in the Jepsen survey. Reading a recent blog article about how the Ceph monitor services have re-implemented their use of Paxis struck me as being extraordinarily complicated. "One Paxos to rule them all"? Oh dear.

I'm doing a back-of-the-envelope examination but I think I already know the answer: you're not going to get a write acknowledged until it is durably stored — which is Consistency. Ceph is a complex system, and parts of it can be offline when others are continuing to provide service. So you'd have to break it down to the provision of a single piece of mutable data before you could study the Availability of the system properly. I'd love to find someone who would like us do a real analysis using the Jepsen techniques; be interesting to see.

But this all reminds us why we're interested in CRDTs in the first place: systems where you can build synchronous communication (or an external appearance thereof care of the use of consensus protocols internally) to achieve Consistency are in essence limited to highly controlled clusters in an individual data center. Most real world systems involve components distributed across geographic, temporal, and logical distances, and that means you must take into account the limitations of the speed of information propagation. While most people immediately think about the light-speed problem, it applies just as much to any distributed environment; and in any real world information system we need to serve clients concurrently, and that means the technique of using a CRDT where possible might very well be worth the effort.

AfC

I think it’s always good to be working on two things: The next most important thing, and the next most interesting thing.

– Jason Fried

Permalink

First the fun...

http://www.huffingtonpost.co.uk/2013/04/24/mars-rover-penis-nasa_n_3144656.html?utm_hp_ref=mostpopular

http://www.smh.com.au/world/strangebuttrue/thieves-steal-55-tonnes-of-nutella-20130409-2hiqg.html

http://www.npr.org/2012/12/01/166293306/the-onion-so-funny-it-makes-us-cry

http://www.theonion.com/articles/kim-jongun-named-the-onions-sexiest-man-alive-for,30379/

http://www.telegraph.co.uk/news/worldnews/middleeast/saudiarabia/10019755/Is-this-man-too-sexy-for-Saudi-Arabia.html

http://www.foreignpolicy.com/articles/2013/04/29/7_things_north_korea_is_really_good_at

http://www.csmonitor.com/World/Middle-East/2013/0515/KFC-smugglers-bring-buckets-of-chicken-through-Gaza-tunnels

http://www.thelocal.es/20130604/spanish-town-mails-dog-poo-back-to-owners

http://www.theaustralian.com.au/news/the-trouble-with-being-kevin-rudd/story-e6frg6n6-1226664160314

http://www.heraldsun.com.au/news/weird/my-erection-wouldnt-deflate-for-240-days/story-fni0cre6-1226664275998 



Now the bugs...

Sega's/Sports Interactive's Football Manager 2009

- look around you'll find further details. Apparently, a known but unfixed bug. Seems as though there may be some hard limits built into the program. Even if you have more than sufficient physical RAM you'll still get into trouble. A few dialog boxes with the following text,"Football Manager 2009 is running dangerously low on memory. Please quit and free up some memory." and "Football Manager 2009 has run out of memory and will now quit."

- need a better way to change contract offer during negotiations especially if another club makes a bid



OpenOffice/LibreOffice

- sometimes not preserving cross references for chapters in text

- consistent issues autocompletion/autoformatting across the board. Example of this is following hyperlink.

http://www.dailytimes.com.pk/default.asp?page=2013\02\17\story_17-2-2013_pg4_7 -->

http://www.dailytimes.com.pk/default.asp?page=2013\02\17\story_17-2-2013



Squid

- requires modification in "init.d" script. Notice lack of whitespace between "squidCreating"? 

user@system:/media/sdc1$ sudo squid restart

user@system:/media/sdc1$ sudo service squid restart

Restarting Squid HTTP proxy: squidCreating squid cache structure ... (warning).

2013/02/16 23:59:42| Creating Swap Directories





Indian Express Website

- too technology dependent. Linking to other pages works fine JavaScript on but when off end up with permission problems (I'm using NoScript add-on)...

http://www.indianexpress.com/news/us-close-to-ok-on-arming-syrian-rebels-reports/1127205/ ->

http://www.indianexpress.com/news/us-close-to-ok-on-arming-syrian-rebels-reports/1127205/2

Forbidden

You don't have permission to access /news/us-close-to-ok-on-arming-syrian-rebels-

reports/1127205/2 on this server.



Wget

- not really a bug but it clearly requires a more informative error message. If file system doesn't support greater than 4GB (FAT32) you end up with the following...

wget

user@system:/media/location/Downloads/CentOS$ wget -c "http://mirror.aarnet.edu.au/pub/centos/6.4/isos/x86_64/CentOS-6.4-x86_64-bin-DVD1.iso"

...

Cannot write to `CentOS-6.4-x86_64-bin-DVD1.iso' (No such file or directory).



Commonwealth Bank ATM

- this was bizarre one. I actually came across an ATM with an IE scripting error problem a while back. I didn't quite believe my eyes so I took a picture of it. Perhaps I'll show it someday?



Google News

- sometimes branching is slow to replicate across their network. Means you end up with orphan/non-existent links from time to time. First time I've encountered this with Google News...



Google Maps

- after Apple's recent problems with their maps service someone in the media said that we shouldn't become overly reliant on such services. I concur. I recently went on a trip to a distant area and had some minor problems. Luckily, i carried a map with me...





Google Translate

- didn't realise it wasn't able to deal with HTTPS well. Ended up with URL invalid errors. Apparently, it's a known problem though...

http://ubuntuincident.wordpress.com/2011/02/07/patch-to-google-translate-https-pages/



Curious why they don't just run it via a proxy such as the following?

http://www.proxyssl.org/

Don't think it would be too hard to replicate given the right modifications?

This post is obviously a continuation of some work in my 'Convergence' report as well as some other blog posts. http://dtbnguyen.blogspot.com.au/2013/05/europes-road-towards-regrowth-part-3.html

  - one of the things which really seperated/distinguished Europe from other countries which I noticed when I was younger was that is somehow able to meld it's history/culture with technological progress. Over time though, it became clear that the technological gap was slowly disappearing, it's edge was somewhat lost and there was not enough progress in other areas. Basic things like healthcare, aged care, social welfare, malfunctioning legal system (across many countries), running a business, even living were needlessly and hopelessly difficult when compared to other countries (including Australia). The overlay of European policy on top of state policy made things even more nonsensical. I think it's about time we push the concept of "power sharing" even further within Europe. This means that we will abolish local (or else European) policies when possible/applicable. We regulate to ensure a safe and fair environment but not enough to inhibit the lives of ordinary citizens and businesses.

http://www.timesofmalta.com/articles/view/20130613/opinion/The-lessons-from-Greece.473645

http://www.nytimes.com/2013/06/12/world/europe/greece.html?ref=world

http://au.news.yahoo.com/world/a/-/world/17611265/eurozone-officials-endorse-3-3-billion-euro-tranche-for-greece-sources/

http://www.ft.com/cms/s/0/c045be3a-cb6a-11e2-b1c8-00144feab7de.html

http://www.independent.co.uk/news/uk/politics/exclusive-the-agricultural-revolution--uk-pushes-europe-to-embrace-gm-crops-8654595.html

http://www.guardian.co.uk/environment/2013/jun/12/gm-crops-environment-secretary-relaxation-rules

http://news.sciencemag.org/scienceinsider/2013/06/auditors-slam-red-tape-at-eu-sci.html

http://phys.org/news/2013-06-eu-cars-dial-case.html

http://phys.org/news/2013-04-aeroplane-lorries-europe.html

http://www.independent.ie/world-news/europe/merkel-says-eu-commission-should-not-get-more-powers-29314248.html

- the same policy should apply to internal state operation as well. If we can gain benefits from mergers, breakups, privatisation, nationalisation, or even complete abolition of departments then we should consider them

- projects like Concorde, Eurotunnel, Airbus A380/A350, ESA's Arianne rocket, CERN's LHC, all prove that Europe can compete with every single other nation on planet with regards to innovation, science, engineering. What's clear though is that that doesn't necessarily translate to real world profitability in a lot of cases. Think about stealth technology, the Internet, even the concepts behind Google and you immediately think about the United States. However, it was clear that Europeans (and other nations) were right in the thick of it from the very beginning...

http://en.wikipedia.org/wiki/Massimo_Marchiori

http://www.webpronews.com/volunia-google-italian-2012-02

http://en.wikipedia.org/wiki/PageRank

http://www.businessweek.com/articles/2013-06-13/france-wants-the-profits-from-french-innovation

http://en.wikipedia.org/wiki/Internet

Several things need to occur if Europe is to profit from it's intellectual prowess; life has to be easier for those who want to start or run a business, got to be easier to find funding in Europe, they need to be willing to take a chance (there have been some serious problems with some major defense projects but it's clear that this occurs with all nations not just Europe. Europe just needs to learn and push on where/when the benefits possibly outweigh the risks in the medium to long term), etc... http://en.wikipedia.org/wiki/Stealth_aircraft

http://en.wikipedia.org/wiki/Horten_Ho_229

http://www.rudebaguette.com/2013/01/23/3-myths-busted-about-european-vs-us-venture-capital-success/

http://www.ft.com/cms/s/0/e80e9b4c-7415-11e1-bcec-00144feab49a.html#axzz2WCMCOopD

- when I meant closer co-operation of European states I meant on many levels. Part of this involved logistics, transporation, education, control of supply/demand, as well as energy. Basically, if/when possible we should subsidise businesess to move within states so that they can be located closer to the sources of their components (manufacturing in particular). This would reduce the cost of transporation, etc... and hopefully drive down the costs of the product resulting in a more competitive option for consumers. It also means that states will need to figure out their place within the union and play their part as part of a co-ordinated European economic trade union. One in which, certain states would specialise in particular areas while others would supplement them via supply chain, raw materials, or else act as a consumer for them. If they can not compete then obviously their status could be challenged by other states who believe that they could do a better job

- as indicated previously, one of the things I've found interesting is how much more dependent on emerging markets (and fringe, smaller, newer European states as well) European states have become for growth. Another thing that I've found curious is the apparent differentcebetween the GDP on many of the poorer states as opposed to those in the stronger states. What I'm wondering is whether or not we should fund investment in these weaker countries instaed of simply outsourcing to Asia/Africa as seems to be the current thinking (is the wage gap large enough to be profitable? are the benefits of the union and reduced transporation/logistical difficulty enough? Should we have a policy of preferecning local, Europe, then rest of globe whenever/ever possible and competitive?). In the longer term, if the weaker states become stronger there would obviously be a larger consumer market within the union as well...

http://edition.cnn.com/2013/02/07/business/diageo-walsh-uk-eu/index.html?iid=article_sidebar

http://data.worldbank.org/indicator/NY.GDP.PCAP.CD

http://data.worldbank.org/indicator/NY.GDP.MKTP.CD

http://www.guardian.co.uk/world/2013/jun/11/latvia-eurozone-membership-economic-growth

http://www.tirebusiness.com/article/20130610/NEWS/130619988/michelin-to-invest-1b-in-france http://www.telegraph.co.uk/finance/comment/ambroseevans_pritchard/10102008/Emerging-markets-displace-Europe-as-fulcrum-of-world-risk.html

The other obvious issue here is if if reducing trade barriers to possible growth markets (or even expansion of the union itself) is one way to achieving growth then the obvious option is to simply expand the union (to increase the size of the possible markets that they can access), else pursue "win-win" trade agreements as they have been doing with the United States, Canada, and so on...

http://www.reuters.com/article/2013/06/13/us-usa-eu-financialservices-idUSBRE95C14420130613

- I think that one thing we all need to face is that Europe has sort of meandered for a while now. Is it that the EU has become less relevant? Are the products/services that they produce simply less competitive?

Table 5: Extra EU-27 trade by main trading partners, EU-27, 2001-2011 (1)

http://epp.eurostat.ec.europa.eu/statistics_explained/index.php/International_trade_in_goods

- for the immediate future the state of the EU will continue to remain the same. It's clear that a "European Superstate" is likely to only occur if countries are willing to take responsibility of their own affairs, others will only help to a limited extent, there continues to remain economic and other stability issues. Moreover with the diverging interests of many countries within the union, it's likely going to be a long time before we see a genuine political union. I suspect it will be a few decades before we may see this and the only way that it is likely to occur is if we go with a flexible, multi-tiered architecture. Already there are 30 odd countries in the union and we've seen in the United States that depending on the circumstances deadlock can result depending on the nature of the political union and the players involved. Yes, we can change the structure but it this will likely involve many compromises for members on a regular basis. The other choice is a change in the structure.

- structure allows for choice between level of "Europeaness". At this current moment in time so many countries with divergent perspectives. Moreover, it is clear that while many European citizens believe in the benefits of the trade union they aren't necessarily in favour of a "superstate". This will provide the structure for a modern union which provides for greater flexibility while also maintaining the benefits of the union itself (tiers can of course be added or removed at will if need be)

Tier 1 - EMU including political union, basically a superstate (economics, military, foreign affairs, etc...) which is similar to the way in which the "United States" is currently setup.

Tier 2 - EMU (basically similar to what we have in the current setup), high number of EU policies required to be implemented per year (percentage based?). Have the right to say yes, no to policies when desired.

Tier 3 - EU trade union but retain currency, medium number of EU policies required to be implemented per year (percentage based?). Have the right to say yes, no to policies when desired.

Tier 4 - EU trade union but retain currency, only small number of EU policies required to be implemented per year (percentage based?). Have the right to say yes, no to policies when desired.

http://euobserver.com/institutional/120484  http://www.politics.co.uk/news/2013/05/31/hague-wants-commons-veto-on-eu-law

http://blogs.spectator.co.uk/mesynon/2013/06/why-william-hagues-red-card-plan-wont-work/

http://au.news.yahoo.com/world/a/-/world/17417763/eu-exit-would-put-britain-on-par-with-norway-danny-alexander/

- I think the debate regarding executive wages is a bit too simplistic. While a cap on bonuses makes sense there are many ways around this as is already been seen/discovered. In my version I would have a formula that is based on a combination of a multiple of the wage of the lowest full time employee plus a variation/combination of factors with regards to overall health of the company. If we can agree bounds on either end (includes everything such as basic wage, salary, bonus, package, incentives...) then we can ensure that they are paid a fair wage while still remaining globally competitive. Moreover, by adding in a time based factor for both performance during his time and a short period after he leaves it gives them greater incentive to work for the longer term benefit for the firm, proper handover procedures, developing a sucession plan and so on... Would like to see an end to so called "golden handshakes" (underperforming executives are sometimes paid enormous amounts for having their employment terminated early). It doesn't make sense to reward incompetence, sub-standard performance, etc...

http://www.ibtimes.co.uk/articles/475363/20130606/banker-bonus-executive-compensation-barclays-rbs-hsbc.htm

http://www.bloomberg.com/news/2013-06-10/most-banks-expect-salary-increases-to-offset-eu-bonus-cap.html

http://www.bloomberg.com/news/2013-06-04/european-parliament-to-delay-vote-on-fund-manager-bonus-rules.html

Would only allow abnormally high renumeration where performance warrants. To deal with issue of massive bias of "institutional voters" would weight votes in such a way that voting rights of both retail and wholesale investors have an equal say. In abscence of vote, abstain.

- as I've stated here and elsewhere I think we hold ourselves back a lot of the time. I'd like to Europe (and the rest of the world) truly chase after

- European leadership need to be clear on what they are presenting to their people and why the changes that they are making are for the "greater good" (if they can't explain why this is the case though I question whether or not they are pursuing the right policy)? At the same time, they need to be representative of their interests. If they are having trouble with passing difficult changes then possibly consider extending term lengths? Think about pursuing a mix of more short term/easy win policies with more difficult medium to longer term structural changes? Other things they need to consider is that their reign is unlikely to last forever. It's possible that some of the changes that they make will not take effect until after they leave office. They might as well as well try to do the "right thing" while they have the chance.

http://www.sbs.com.au/news/article/1777699/Comment--Rudd,-Gillard-or-Abbott---Do-leaders-really-matter-

Moreover, it's not just what they say but how they say it as well. Try to explain to them that there basically is not alternative apart from reform. The type/level of reform can of course be negotiated but the other choice is to basically let the state go bankrupt and have chaos ensue. Moreover, what Europeans need to understand is that without reform their countries and Europe will cease to become relevant. Trade is trending down between many EU nations as is...

http://www.irishtimes.com/business/double-dutch-loopholes-get-multinationals-off-tax-hook-1.1411926

It would also help if respsonsibility for the dealing with the problem was better targted (though this has changed of late)...

- the European leadership have probably done a poor job in commmunicating why some of the measures were required, why they were necessary in order to achieve stability within the Eurozone, why existing the Eurozone would have been so dangerous, etc... An example of this is explaining the purpose of taxes within society (especially int the context of Greece). If you've never paid tax then it is unlikely that you will wan to pay it. After all, if you don't make use of social services why would you have any reason to use them? The most amusing explanation of this that I've come across would be the notion of a "TV Collector Man". If for generations someone regularly took your TV from your home you would gradually come to accept it but if this were suddenly imposed upon you you would consider it a form of theivery.

http://www.telegraph.co.uk/finance/comment/10099326/Theres-a-worse-crisis-on-the-way-unless-we-get-serious-about-tackling-debt.html#disqus_thread

I think a better way to explain it to them is that without taxes the whole public sector would basically collapse.

- for every person who criticises the European leadership/Troika over some of the policies that they have pursued they need to ask themselves just exactly what would they have done in the same circumstances. Very few people are trained/qualified to run a large company, fewer still countries, and possibly no one who has a career/history in dealing with bankrupt states.

- labour mobility has been an option outlined here an elsewhere as a means of dealing with the employment crisis. One thing I've been wondering is whether or not there should be "core languages" that should be taught across the union? Based on what I know many countries already teach secondary languages to students as part of their education systems so implementation shouldn't be too foreign to member states...

http://www.ft.com/cms/s/0/b210ec48-ca1d-11e2-af47-00144feab7de.html

http://www.slate.com/blogs/moneybox/2013/06/12/eurozone_internal_migration.html

http://www.telegraph.co.uk/finance/jobs/10097363/Come-to-Germany-to-work-and-find-love-British-are-told.html

- I think that while the protesters from the "Occupy/Blockupy" movements have clear grievances they'll never gain traction unless they clarify their message, attempt to come up with a solutions to the problems that they are describing, or possibly even attempt to change the system from within?

http://www.huffingtonpost.com/nathan-gardels/europe-a-leaning-tower-of_b_3366506.html

http://www.thetrumpet.com/article/10697.19.0.0/economy/thousands-protest-austerity-in-europe

http://www.thetrumpet.com/article/10611.29392.0.0/world/government/europes-unemployed-an-army-waiting-for-a-leader

http://www.iol.co.za/business/international/all-the-people-in-europe-will-demonstrate-1.1525639

http://www.telegraph.co.uk/news/worldnews/europe/germany/10093485/Scuffles-as-Germany-pushes-back-against-austerity.html

- budget for European administration is huge. Should look at savings within union structure itself and devolve savings back into countries that are in trouble. Are we sure we need so many MEPs? Should we consider a freeze on wage increases/hiring? Consolidate to a single base in Brussels or else reduce the importance of other locations (savings of several hundred million per year possible apparently)? Increased savings through better use of technology to flesh out details of major European policy before meetings occur? Currently, discussions are getting strung out for too long...

http://www.europarl.org.uk/view/en/infocentre/faqs.html

http://www.europarl.europa.eu/aboutparliament/en/0081ddfaa4/MEPs.html

http://en.wikipedia.org/wiki/European_Parliament

http://www.euractiv.com/future-eu/salaries-eu-officials-increase-n-news-516759

http://www.telegraph.co.uk/news/worldnews/europe/9696232/David-Cameron-Eurocrats-must-accept-cuts-to-pay-and-perks.html

http://openeuropeblog.blogspot.com.au/2013/02/should-we-feel-sorry-for-underpaid-eu.html

http://www.europarl.europa.eu/news/en/headlines/content/20121211FCS04528/8/html/Would-you-prefer-to-be-EU-or-UK-civil-servant

- are they banks being realistic in attemping to get some bad assets off of their balance sheets? Even in better economic conditions it's clear that it will take some time before they can get rid of these assets. Should they just accept that they will take the a hit to their earnings in the short term? Should we give banks in the weaker countries more/less time to deal with their problems (basically giving them a temporary competitive ad/disadvantage/setting up a regulatory arbitrage situation)?

http://online.wsj.com/article/SB10001424127887323734304578542262346050342.html

http://www.ft.com/cms/s/0/54e9beea-d0ef-11e2-be7b-00144feab7de.html

http://www.chronicle.gi/headlines_details.php?id=29534

- I don't see the EU as just a trade block. I see it as much more than that. In the future I see a multi-polar world prevailing and it is likely that Europe become be a part of it (as long as it is able to deal with it's present problems). These trade blocks will form the basis for sharing of burdens and responsibilities for global issues and these blocks will act in the interests of all nations within that block as well as the interests of that particular block within the world. Basically, the notion of notion of nationally based superpowers will have far less of an impact if these blocks are able to maintain solidarity.

- it's clear that the Eurozone has sort of stablised (there's still clearly a lot of other clean up work occurring especially with regards to the banking sector).

http://www.latimes.com/news/nationworld/world/la-fg-europe-austerity-20130615,0,2953904.story

http://www.euronews.com/2013/06/14/growth-the-key-in-europe-says-portuguese-president/

https://en.wikipedia.org/wiki/Marshall_Plan

http://www.theaustralian.com.au/business/even-greece-sees-light-at-the-end-of-the-tunnel/story-e6frg8zx-1226664130435

http://www.cyprus-mail.com/cyprus/cyprus-signs-100m-loan-agreement-eib/20130531

http://www.theprovince.com/news/Commentary+Italian+showdown+with+Germany+over+eurozone+looms/8517825/story.html

http://www.bloomberg.com/news/2013-06-04/draghi-covets-bank-clean-up-as-ecb-weighs-fix-to-loan-dro.html

This means that instead of having to focus our resources on stabilising we can focus more on the process of using it to stimulate growth. From now on (whereever possible), I propose that every single dollar that can be saved at the European or nation level will be counter-blanced by funds for the direct funding growth initiatives such as developpment of businesses, better infrastructure (problem is that most infrastructure in Europe is reasonably modern and well integrated. Not too many projects that would be major game changers (energy sector and perhaps better integration of transportation/logistics with states that are newly entering the union to facilitate trade), etc... in troubled states? Would like to see greater input/discussion from various stakeholders on what they believe to be imperative to future of Europe...), etc...

http://www.itwire.com/it-industry-news/strategy/60283-will-maths-help-solve-melbourne%E2%80%99s-transport-problems?

http://www.dw.de/strikes-paralyze-rail-system-in-france/a-16877911

http://www.timesofmalta.com/articles/view/20130613/opinion/The-lessons-from-Greece.473645

http://au.news.yahoo.com/world/a/-/world/17462432/france-ireland-eyeing-submarine-power-cable-link/

http://www.independent.ie/irish-news/energy-linkup-to-france-proposed-29315855.html

http://www.bloomberg.com/news/2013-06-11/croatia-joins-greece-italy-to-push-for-europe-caspian-gas-link.html

- unrealistic that we abolish tax havens, etc... in the near term but we can at least reduce the spread (especially with regards to exceptionally low corporate tax rates). This will allow these countries to maintain their deposits/competitive advantage but we could use these funds/savings to help deal with some of the more troubled states where possible...

http://www.nytimes.com/2013/06/15/opinion/a-chance-to-do-better-on-greece.html

- Europe really needs to look hard at itself. Figure out what it's citizens want with regards to welfare, what it can pay for, and how much they are willing to pay in order to get these particular services. I recall high income professionals in parts of Europe who were paying atrociously high tax rates to the point where I often wondered whether there was a point of diminishing returns...

http://www.smh.com.au/opinion/politics/how-an-abbott-government-may-run-the-economy-20130611-2o1yz.html

http://www.finance.gov.au/archive/archive-of-publications/ncoa/coaintro.htm

That's the basis from which I think we should be thinking about cuts, not necessarily just cutting without necessarily understanding the real world implications... which brings me to the next point. I think there may be a slight disconnect between what they are feeling and what is actually happening on the ground. Recently, a few local politicians tried living on welfare benefits for a short period of time to see what is was like. I challenge some members of the European leadership to do the same. Figure out just how far you can push without taking people "over the edge" and don't be afraid of rolling back changes if they don't have the desired effect.

http://www.guardian.co.uk/commentisfree/2013/jun/09/bedroom-tax-huge-problems-worse

http://www.bbc.co.uk/democracylive/scotland-22886745

- I think sometimes European states think about the problem of dealing with their deficits too simplistically. Sometimes it seems as though they just think about cuts and revenue raising through increased taxes (aware that this is not always the case). Europe shouldn't be afraid fund businesses directly or help companies grow larger... if they are successful that means that the size of their tax base instantly increases (of course starting a business is never easy)

http://epp.eurostat.ec.europa.eu/statistics_explained/index.php/Business_economy_-_size_class_analysis

http://www.euro2day.gr/ftcom_en/article-ft-en/1103190/uk-funding-for-lending-scheme-fails-to-spur-credit.html

http://www.theportugalnews.com/news/germany-prepares-development-bank-aid-for-portugal-spain/28547

http://www.ibtimes.com/french-airport-strikes-could-expand-10-eu-nations-wednesday-1302811

http://www.euronews.com/2013/06/12/french-air-traffic-control-strike-2013-latest-news

http://www.irishtimes.com/news/chaos-across-europe-as-french-air-traffic-controllers-ground-flights-1.1426321

http://www.businessspectator.com.au/news/2013/6/13/renewable-energy/europe-must-act-make-green-desert-project-work-desertec-head

- one of the things I've found interesting when looking at various statistics regarding Europe is how disperse the spread is with regards to the size of businesses and also the size of government spending/taxation in relation to overall GDP. What's noticeable is how healthier econonomies tend to spread the load more evenly across the private/public sector as well as across businesses of varying sizes...

http://epp.eurostat.ec.europa.eu/statistics_explained/index.php/Business_economy_-_size_class_analysis

https://en.wikipedia.org/wiki/Government_spending

http://data.worldbank.org/indicator/NE.CON.GOVT.ZS

http://www.guardian.co.uk/commentisfree/2013/jun/13/coalitions-austerity-reality-british-politics

http://www.timesofmalta.com/articles/view/20130613/opinion/The-lessons-from-Greece.473645

- capitalism and other social models in my mind are simply a highly evolved version of Darwinism within the framework of human behaviour which is then overlaid upon a series of theoretical manifestations which allow us to live in a relatively harmonious fashion. However, over time it's become clear that huge abberations/distortions have occurred which have led to a skew in the overall ecosystem which have meant that system needs to be altered. We've hit the point at which MNC/TNC powers have almost superceded that of countries and perhaps even continents. Clearly, this can skew government decision making since we've become so dependent on them for employment and income from them. The biggest problem is this. If our governments are basically being run by people who are looking after the interests of business rather that of their citizens then everybody loses. If governments are inefficient or even bankrupt we have political/social unrest, if businesses hold too much sway then eventually they may one day end up running many government services. Something which is often unprofitable and probably wouldn't be the workeable in the long term. This is what everyone should keep in mind if we continue down the pure corporate path that some nations seems to be taking (I highly doubt an executive at a software company is going to be interested/skilled in running government and vice-versa). Everyone has a role to play...

http://www.reuters.com/article/2013/06/12/spain-esm-idUSL5N0EO2OV20130612

http://www.ft.com/cms/s/0/e5df7096-d3f8-11e2-8639-00144feab7de.html

http://www.omfif.org/

http://en.wikipedia.org/wiki/Official_Monetary_and_Financial_Institutions_Forum

This is about getting what's best for everyone while still being able to maintain our belief and values that we work within a meritocracy. as long as they control our world what do we have to look forward to?

http://www.guardian.co.uk/commentisfree/2013/jun/13/ineffectual-g8-protests-stagnant-politics

http://www.presstv.ir/detail/2013/06/01/306653/germany-prosperity-relies-on-rest-of-eu/

- If you can't change their behaviour change the system. The thing that needs to be explained here is that we don't really care how much a company makes as long as they pay a fair share of tax. They are dependent on us to make their income just as we are reliant on them for employment

http://www.telegraph.co.uk/news/politics/10117682/David-Cameron-must-intervene-in-Google-tax-row-says-Margaret-Hodge.html

http://www.guardian.co.uk/world/2013/jun/04/david-cameron-tax-crackdown-g8

http://www.businessspectator.com.au/comment/318926

http://krugman.blogs.nytimes.com/2013/06/03/ben-bernanke-endorses-a-73-percent-tax-rate/

http://www.epi.org/publication/raising-income-taxes/

http://news.cnet.com/8301-13579_3-57587335-37/apple-owes-france-$6.5-million-in-unpaid-taxes/

http://au.news.yahoo.com/latest/a/-/latest/17510588/france-targets-multinationals-with-tax-rules-shake-up/

http://www.telegraph.co.uk/finance/financialcrisis/10040464/IMF-tells-Greece-to-step-up-fight-on-tax-evasion.html

- the problem is that "protected" industries don't learn to adapt and change to suit the circumstances. For instance, for a long while now local car companies have insisted on making large cars even though the trend is clearly moving towards smaller, more fuel efficient vehicles. The concern is that the more "protection" you have the more likely it is that you become complacent, don't adapt to modernisation, global best practices, or become overly dependent on government subsidies

http://www.skynews.com.au/topstories/article.aspx?id=874831

http://www.theaustralian.com.au/national-affairs/carmaker-knew-end-had-come-two-years-ago/story-fn59niix-1226658124392

- focus on 'real value' in products. Not just incremental changes but substantial improvements over current science technology. Incremental improvements provide easy sales when there is a relatively solid, known market. To make real gains you need to be bold and innovative

- states don't need to take advice from the European administration but they need to heed the warnings and realise that in many cases there is a lot of expertise and help available from within the union should they require. Moreover, it's clear much of Europe looking at similar problems with regards to dealing with excessive debt... States could learn from one another's experiences... To deal with the credibility issue, have three levels national review, European review, and private review (similar to the methodolgy envisaged for bank supervision by Mario Draghi)

http://en.wikipedia.org/wiki/Irish_property_bubble

http://www.guardian.co.uk/world/2013/jun/13/hollande-french-pensions-system-overhaul

http://www.hrreporter.com/articleview/18214-workers-in-france-should-pay-into-pensions-for-longer-panel

http://www.thelocal.fr/20130614/eu-orders-france-to-reform-pensions-and-jobs-market

http://online.wsj.com/article/SB10001424127887324688404578545692498519664.html

http://www.bloomberg.com/news/2013-06-12/hollande-to-ask-french-to-work-more-as-pension-deficit-balloons.html

Too strict/loose with regards to healthcare contributions?

http://en.wikipedia.org/wiki/Pensions_in_France

http://www.cleiss.fr/docs/regimes/regime_france/an_3.html

http://en.wikipedia.org/wiki/Social_Security_in_France

http://en.wikipedia.org/wiki/Health_care_in_France

If you look hard enough you'll notice that the solutions are already provided by those who are already in the industry already. Need to factor in the zoom in/zoom out issue. All you have to do is ask the people in question what is wrong with the system and it's more than likely that they'll already have the answer for you

- it seems as though there is endless complaints about how slow the change takes to complete within the union. This is true but they also need to realise that it still isn't a "superstate" as yet and is unlikely to be so in the near future unless structural of philosophical changes occur

http://www.businessweek.com/news/2013-06-04/draghi-covets-bank-clean-up-as-ecb-weighs-fix-to-loan-drought

http://www.breakingviews.com/hugo-dixon-why-draghi-likes-london/21087903.article

http://www.independent.ie/opinion/analysis/colm-mccarthy-us-puts-europe-to-shame-in-dealing-with-recession-29313937.html

- someday I'd love to have a future in which money is no longer relevant and we can basically do whatever we want with our lives but as it stands we have to make do with what we have. Altering the system to suit our needs rather than working with a system that serves only a small minority must surely be in the best interests of everyone?



http://www.social-europe.eu/2013/03/germany-has-created-an-accidental-empire/

http://www.reuters.com/article/2013/06/13/us-g8-russia-putin-idUSBRE95C18N20130613

http://www.bloomberg.com/news/2013-06-14/singapore-censures-20-banks-for-attempts-to-rig-benchmark-rates.html

http://www.brisbanetimes.com.au/business/world-business/why-abenomics-will-work-20130411-2hmvh.html

http://www.icij.org/offshore/secret-files-expose-offshores-global-impact

http://news.sciencemag.org/scienceinsider/2013/05/frances-message-to-science-help-.html

http://www.economist.com/news/europe/21578656-germanys-vaunted-dual-education-system-its-latest-export-hit-ein-neuer-deal

http://www.dailymail.co.uk/news/article-2335479/Germany-recruit-British-apprentices-Work-study-offer-lure-brightest-youngsters.html?ito=feeds-newsxml

http://www.telegraph.co.uk/finance/economics/10097303/Black-market-is-10pc-of-UK-economy-says-IEA.html

http://www.washingtontimes.com/news/2013/jun/3/our-uncompetitive-economy/

http://www.express.co.uk/comment/columnists/leo-mckinstry/407173/We-just-can-t-afford-the-welfare-bill-for-eurozone-migrants

http://manifest-europa.eu/allgemein/wir-sind-europa?lang=en

http://www.economist.com/news/leaders/21579456-if-europes-economies-are-recover-germany-must-start-lead-reluctant-hegemon

http://www.guardian.co.uk/global-development/2013/jun/12/european-union-laws-extractive-industries-payments

http://epp.eurostat.ec.europa.eu/statistics_explained/index.php/Tourism_trends

http://epp.eurostat.ec.europa.eu/statistics_explained/index.php/Tourism_statistics_at_regional_level

http://www.guardian.co.uk/business/2013/jun/12/bond-bubble-threatens-financial-system

http://www.irishtimes.com/business/sectors/financial-services/eu-countries-disguising-true-financial-position-1.1427130

http://epp.eurostat.ec.europa.eu/statistics_explained/index.php/Population_and_population_change_statistics

http://www.telegraph.co.uk/finance/libor-scandal/10102025/London-threatened-by-plans-to-move-Libor-regulation-to-Paris.html

http://www.bloomberg.com/news/2013-06-03/spain-s-crisis-fades-as-exports-lead-the-way.html

http://epp.eurostat.ec.europa.eu/statistics_explained/index.php/International_trade_by_enterprise_characteristics

http://www.social-europe.eu/2013/03/germany-has-created-an-accidental-empire/

http://www.businessinsider.com/richard-koo-the-entire-crisis-in-europe-started-with-a-big-ecb-bailout-of-germany-2012-6#ixzz1yIahXEli

http://www.social-europe.eu/2013/06/schroder-tells-france-to-assume-germanys-role-of-exploding-the-euro-area/

http://wallstcheatsheet.com/economy/europe-showdown-will-germany-affirm-ecbs-super-solution.html/?a=viewall

http://www.guardian.co.uk/business/2013/jun/09/eurozone-crisis-debt-income-ratios

http://www.guardian.co.uk/world/2013/jun/01/germany-champion-europe

http://www.bloomberg.com/news/2013-06-01/van-rompuy-direction-of-budget-reforms-more-important.html

http://economictimes.indiatimes.com/news/international-business/european-union-to-pave-way-for-class-actions-against-cartels/articleshow/20467305.cmshttp://theprodigalgreek.wordpress.com/2013/06/06/honey-i-shrunk-the-greeks/

http://qz.com/91617/portugal-is-poorer-today-than-at-the-start-of-the-century/

http://www.washingtonpost.com/blogs/wonkblog/wp/2013/06/06/how-much-blame-does-mervyn-king-deserve-for-britains-broken-economy/ http://www.telegraph.co.uk/news/worldnews/asia/china/10103374/Is-Chinas-Xi-Jinping-ready-to-do-business-with-the-US.html

To submit your proposal, create an account, and select Submit a Miniconf from the menu on the left hand side.

Important Dates

• Call for miniconfs opens: 14 June 2013
• Call for miniconfs closes: 14 July 2013
• Email notifications from papers committee: 28 August 2013
• Conference dates: Monday 6 January to Friday 10 January 2014
• Miniconfs run: Monday 6 January to Tuesday 7 January 2014

Information on Proposals

The linux.conf.au 2014 papers committee is looking for a broad range of proposals, and will consider submissions on anything from programming and software, to desktop, userspace, community, government, and education. There is only one rule:

Your proposal must be related to open source

This year, the papers committee is going to be focused on deep technical content, and things we think are going to really matter in the future -- that might range from freedom and privacy to open source cloud systems or to energy efficient server farms of the future.

In the past, we have held miniconferences on the following topics:

• Haecksen
• Libré Graphics
• Women in open source
• Business in open source
• Open source in education
• Systems Administration

Audio Visual PlUS many more

LCA is known for miniconfs that are strongly technical in nature, but proposals for presentations on other aspects of free software and open culture, such as educational and cultural applications of open source, are welcome.

Code of Conduct

linux.conf.au welcomes first-time and seasoned speakers from all free and open communities - people of all ages, genders, nationalities, ethnicities, backgrounds, religions, abilities, and walks of life. We respect and encourage diversity at our conference.

By agreeing to present at or attend the conference, you are agreeing to abide by the terms and conditions. We expect all speakers and delegates to have read and understood our Code of Conduct.

Format

Miniconfs are day-long sessions on a specific topic. As the name suggests, they are expected to be run as a miniature conference, with a formal schedule published ahead of time listing speakers and sessions for the day.

Your miniconf can have any combination of formal presentations, demonstrations, tutorials, workshops, panel discussions, or other display that you feel would be of interest to a linux.conf.au audience. The only thing we ask is that you keep it contained to your room, that you clean up afterwards, and that you ask permission of the organising committee before arranging anything potentially dangerous (such as soldering, rocketry, or anything involving flames or volatile substances).

Miniconf Organiser Information

In recognition of the value that miniconf organisers bring to our conference, once a proposal is accepted a miniconf organiser is entitled to:

• Free registration for one organiser, which holds all of the benefits of a Professional Delegate Ticket

If your proposal includes more than one miniconf organiser, free registration and any extra benefits are provided to the primary organiser only.

Please note: miniconf speakers do not receive free tickets to the conference. They must purchase their own ticket in order to attend and present at your miniconf. Please communicate this clearly when inviting your potential speakers to your miniconference.

linux.conf.au does not and will not pay speakers (including miniconf speakers) to present at the conference. Similarly, miniconf organisers are not permitted to accept corporate or government sponsorship, nor are they permitted to charge an admittance fee to delegates.

linux.conf.au is able to provide limited financial assistance for some miniconf organisers, for instance, where the cost of flights or accommodation might prohibit you from attending. Please note, however, that there is a limited budget for travel assistance and that asking for assistance could affect your chances of acceptance.

Recording and Licensing

To increase the number of people that can view your miniconference, linux.conf.au might record your miniconf and make it publicly available after the event. Please ensure that your miniconf speakers will be expected to release materials relating to their presentation under a Creative Commons ShareAlike License. Additionally, if your speakers are discussing software in their presentation, the software must have an appropriate open licence.

About Linux Australia

Linux Australia is the peak body for open source communities around Australia, and as such represents approximately 3500 Free and Open Source users and developers. Linux Australia supports the organisation of this international Free Software conference in a different Australasian city each year.

For more information about Linux Australia see www.linux.org.au

Enquiries

Email the linux.conf.au 2014 Papers Committee at papers-chair at lca2014.linux.org.au