You are here


Anthony Towns: Bitcoin: ASICBoost and segwit2x – Background

Planet Linux Australia - Mon 10th Jul 2017 15:07

I’ve been trying to make heads or tails of what the heck is going on in Bitcoin for a while now. I’m not sure I’ve actually made that much progress, but I’ve at least got some thoughts that seem coherent now.

First, this post is background for people playing along at home who aren’t familiar with the issues or jargon: Bitcoin is a currency based on an electronic ledger that essentially tracks how much Bitcoin exists, and how someone can be authorised to transfer it to someone else; that ledger is currently about 100GB in size, growing at a rate of about a gigabyte a week. The ledger is updated by miners, who compete by doing otherwise pointless work running cryptographic hashes (and in so doing obtain a “proof of work”), and in return receive a reward (denominated in bitcoin) made up from fees by people transacting and an inflation subsidy. Different miners are competing in an essentially zero-sum game, because fees and inflation are essentially a fixed amount that is (roughly) divided up amongst miners according to how much work they do — so while you get more reward for doing more work, it comes at a cost of other miners receiving less reward.

Because the ledger only grows by (about) a gigabyte each week (or a megabyte per block, which is roughly every ten minutes), there is a limit on how many transactions can be included each week (ie, supply is limited), which both increases fees and limits adoption — so for quite a while now, people in the bitcoin ecosystem with a focus on growth have wanted to work out ways to increase the transaction rate. Initial proposals in mid 2015 suggested allowing miners to regularly determine the limit with no official upper bound (nominally “BIP100“, though never actually formally submitted as a proposal), or to increase by a factor of eight within six months, then double every two years after that, until reaching almost 200 times the current size by 2036 (BIP101), or to increase at a rate of about 17% per annum (suggested on the mailing list, but never formally proposed). These proposals had two major risks: locking in a lot of growth that may turn out to be unnecessary or actively harmful, and requiring what is called a “hard fork”, which would render the existing bitcoin software unable to track the ledger after the change took affect with the possible outcome that two ledgers would coexist and would in turn cause a range of problems. To reduce the former risk, a minimal compromise proposal was made to “kick the can down the road” and just double the ledger growth rate, then figure out a more permanent solution down the road (BIP102) (or to double it three times — to 2MB, 4MB then 8MB — over four years, per Adam Back). A few months later, some of the devs figured out a way to more or less achieve this that also doesn’t require a hard fork, and comes with a host of other benefits, and proposed an update called “segregated witness” at the December 2015 Scaling Bitcoin conference.

And shortly after that things went completely off the rails, and have stayed that way since. Ultimately there seem to be two camps: one group is happy to deploy segregated witness, and is eager to make further improvements to Bitcoin based on that (this is my take on events); while the other group does not, perhaps due to some combination of being opposed to the segregated witness changes directly, wanting a more direct upgrade immediately, being afraid deploying segregated witness will block other changes, or wanting to take control of the bitcoin codebase/roadmap from the current developers (take this with a grain of salt: these aren’t opinions I share or even find particularly reasonable, so I can’t do them justice when describing them; cf ViaBTC’s post to get that side of the argument made directly, eg)

Most recently, and presumably on the basis that the opposed group are mostly worried that deploying segregated witness will prevent or significantly delay a more direct increase in capacity, a bitcoin venture capitalist, Barry Silbert, organised an agreement amongst a number of companies including many miners, to both activate segregated witness within the next month, and to do a hard fork capacity increase by the end of the year. This is the “segwit2x” project; named because it takes segregated witness, (“segwit”) and then additionally doubles its capacity increase (“2x”). This agreement is not supported by any of the existing dev team, and is being developed by Jeff Garzik (who was behind BIP100 and BIP102 mentioned above) in a forked codebase renamed “btc1“, so if successful, this may also satisfy members of the opposed group motivated by a desire to take control of the bitcoin codebase and roadmap, despite that not being an explicit part of the agreement itself.

To me, the arguments presented for opposing segwit don’t really seem plausible. As far as future development goes, a roadmap was put out in December 2015 and endorsed by many developers that explicitly included a hard fork for increased capacity (“moderate block size increase proposals (such as 2/4/8 …)”), among many other things, so the risk of no further progress happening seems contrary to the facts to me. The core bitcoin devs are extremely capable in my estimation, so replacing them seems a bad idea from the start, but even more than that, they take a notably hands off approach to dictating where Bitcoin will go in future — so, to my mind, it seems like a more sensible thing to try would be working with them to advance the bitcoin ecosystem in whatever direction you want, rather than to try to replace them outright. In that context, it seems particularly notable to me that in the eighteen months between the segregated witness proposal and the segwit2x agreement, there hasn’t been any serious attempt to propose a hard fork capacity increase that meets the core dev’s quality standards; for instance there has never been any code for BIP100, and of the various hard forking codebases that have arisen by advocates of the hard fork approach — Bitcoin XT, Bitcoin Classic, Bitcoin Unlimited, btc1, and Bitcoin ABC — none have been developed in a way that’s suitable for the changes to be reviewed and merged into core via a pull request in the normal fashion. Further, since one of the main criticisms of a hard fork is that deployment costs are higher when it is done in a short time frame (weeks or a few months versus a year or longer), that lack of engagement over the past 18 months followed by a desperate rush now seems particularly poor to me.

A different explanation for the opposition to segwit became public in April, however. ASICBoost is a patent-pending optimisation to the way Bitcoin miners do the work that entitles them to extend the ledger (for which they receive the rewards described earlier), and while there are a few ways of making use of ASICBoost, perhaps the most effective way turns out to be incompatible with segwit. There are three main alternatives to the covert, segwit-incompatible approach, all of which have serious downsides. The first, overt ASICBoost via modifying the block version reveals that you’re using ASICBoost, which would either (a) encourage other miners to also use the optimisation reducing your profits, (b) give the patent holder cause to charge you royalties or cause other problems (assuming the patent is eventually granted and deemed valid), or (c) encourage the bitcoin community at large to change the ecosystem rules so that the optimisation no longer works. The second, mining empty blocks via ASICBoost means you don’t gain any fee income, reducing your revenue and hence profit. And the third, rolling the extranonce to find a collision rather than combining partial transaction trees increases the preparation work by a factor of ten or so, which is probably enough to outweigh the savings from the optimisation in the first place.

If ASICBoost were being used by a significant number of miners, and segregated witness prevents its continued use in practice, then we suddenly have a very plausible explanation for much of the apparent madness: the loss of the optimisation could significantly increase some miners’ costs or reduce their revenue, reducing profit either way (a high end estimate of $100,000,000 per year was given in the original explanation), which would justify significant investment in blocking that change. Further, an honest explanation of the problem would not be feasible, because this would be just as bad as doing the optimisation overtly — it would increase competition, alert the potential patent owners, and might cause the optimisation to be deliberately disabled — all of which would also negatively affect profits. As a result, there would be substantial opposition to segwit, but the reasons presented in public for this opposition would be false, and it would not be surprising if the people presenting these reasons only give half-hearted effort into providing evidence — their purpose is simply to prevent or at least delay segwit, rather than to actually inform or build a new consensus. To this line of thinking the emphasis on lack of communication from core devs or the desire for a hard fork block size increase aren’t the actual goal, so the lack of effort being put into resolving them over the past 18 months from the people complaining about them is no longer surprising.

With that background, I think there are two important questions remaining:

  1. Is it plausible that preventing ASICBoost would actually cost people millions in profit, or is that just an intriguing hypothetical that doesn’t turn out to have much to do with reality?
  2. If preserving ASICBoost is a plausible motivation, what will happen with segwit2x, given that by enabling segregated witness, it does nothing to preserve ASICBoost?

Well, stay tuned…

Categories: thinktime

This week's sponsor: ​Jamf Now

a list apart - Mon 10th Jul 2017 14:07

JAMF NOW is an on-demand mobile device management solution for your Apple devices at work. Get started for free today!

Categories: thinktime

But what if you're doing it wrong?

Seth Godin - Sun 09th Jul 2017 18:07
Marketing doesn't have to suck. It doesn't have to be a miserable experience for consumers, and it certainly doesn't have to be a distasteful, creepy or annoying task for the creator. We don't have to market at people, pin them...        Seth Godin
Categories: thinktime

Lev Lafayette: One Million Jobs for Spartan

Planet Linux Australia - Sat 08th Jul 2017 23:07

Whilst it is a loose metric, our little cluster, "Spartan", at the University of Melbourne ran its 1 millionth job today after almost exactly a year since launch.

The researcher in question is doing their PhD in biochemistry. The project is a childhood asthma study:

"The nasopharynx is a source of microbes associated with acute respiratory illness. Respiratory infection and/ or the asymptomatic colonisation with certain microbes during childhood predispose individuals to the development of asthma.

Using data generated from 16S rRNA sequencing and metagenomic sequencing of nasopharyn samples, we aim to identify which specific microbes and interactions are important in the development of asthma."

Moments like this is why I do HPC.

Congratulations to the rest of the team and to the user community.

read more

Categories: thinktime

Caring is free

Seth Godin - Sat 08th Jul 2017 19:07
In the short run, of course, not caring can save you some money. Don't bother making the facilities quite so clean. Save time and hassle and let the display get a little messy. Don't worry so much about one particular...        Seth Godin
Categories: thinktime

Danielle Madeley: Using the Nitrokey HSM with GPG in macOS

Planet Linux Australia - Fri 07th Jul 2017 19:07

Getting yourself set up in macOS to sign keys using a Nitrokey HSM with gpg is non-trivial. Allegedly (at least some) Nitrokeys are supported by scdaemon (GnuPG’s stand-in abstraction for cryptographic tokens) but it seems that the version of scdaemon in brew doesn’t have support.

However there is gnupg-pkcs11-scd which is a replacement for scdaemon which uses PKCS #11. Unfortunately it’s a bit of a hassle to set up.

There’s a bunch of things you’ll want to install from brew: opensc, gnupg, gnupg-pkcs11-scd, pinentry-mac, openssl and engine_pkcs11.

brew install opensc gnupg gnupg-pkcs11-scd pinentry-mac \ openssl engine-pkcs11

gnupg-pkcs11-scd won’t create keys, so if you’ve not made one already, you need to generate yourself a keypair. Which you can do with pkcs11-tool:

pkcs11-tool --module /usr/local/lib/ -l \ --keypairgen --key-type rsa:2048 \ --id 10 --label 'Danielle Madeley'

The --id can be any hexadecimal id you want. It’s up to you to avoid collisions.

Then you’ll need to generate and sign a self-signed X.509 certificate for this keypair (you’ll need both the PEM form and the DER form):

/usr/local/opt/openssl/bin/openssl << EOF engine -t dynamic \ -pre SO_PATH:/usr/local/lib/engines/ \ -pre ID:pkcs11 \ -pre LIST_ADD:1 \ -pre LOAD \ -pre MODULE_PATH:/usr/local/lib/ req -engine pkcs11 -new -key 0:10 -keyform engine \ -out cert.pem -text -x509 -days 3640 -subj '/CN=Danielle Madeley/' x509 -in cert.pem -out cert.der -outform der EOF

The flag -key 0:10 identifies the token and key id (see above when you created the key) you’re using. If you want to refer to a different token or key id, you can change these.

And import it back into your HSM:

pkcs11-tool --module /usr/local/lib/ -l \ --write-object cert.der --type cert \ --id 10 --label 'Danielle Madeley'

You can then configure gnupg-agent to use gnupg-pkcs11-scd. Edit the file ~/.gnupg/gpg-agent.conf:

scdaemon-program /usr/local/bin/gnupg-pkcs11-scd pinentry-program /usr/local/bin/pinentry-mac

And the file ~./gnupg/gnupg-pkcs11-scd.conf:

providers nitrokey provider-nitrokey-library /usr/local/lib/

gnupg-pkcs11-scd is pretty nifty in that it will throw up a (pin entry) dialog if your token is not available, and is capable of supporting multiple tokens and providers.

Reload gpg-agent:

gpg-agent --server gpg-connect-agent << EOF RELOADAGENT EOF

Check your new agent is working:

gpg --card-status

Get your key handle (grip), which is the 40-character hex string after the phrase KEY-FRIEDNLY (sic):

gpg-agent --server gpg-connect-agent << EOF SCD LEARN EOF

Import this key into gpg as an ‘Existing key’, giving the key grip above:

gpg --expert --full-generate-key

You can now use this key as normal, create sub-keys, etc:

gpg -K /Users/danni/.gnupg/pubring.kbx ------------------------------- sec> rsa2048 2017-07-07 [SCE] 1172FC7B4B5755750C65F9A544B80C280F80807C Card serial no. = 4B43 53233131 uid [ultimate] Danielle Madeley <> echo -n "Hello World" | gpg --armor --clearsign --textmode

Side note: the curses-based pinentry doesn’t deal with piping content into stdin, which is why you want pinentry-mac.

You can also import your certificate into gpgsm:

gpgsm --import < ca-certificate gpgsm --learn-card

And that’s it, now you can sign your git tags with your super-secret private key, or whatever it is you do. Remember that you can’t exfiltrate the secret keys from your HSM in the clear, so if you need a backup you can create a DKEK backup (see the SmartcardHSM docs), or make sure you’ve generated that revocation certificate, or just decided disaster recovery is for dweebs.

Categories: thinktime

The rationality paradox

Seth Godin - Fri 07th Jul 2017 18:07
If you see yourself as an engineer, a scientist, or even a person of logic, then it's entirely possible that you work to make rational decisions, decisions that lead to the outcomes you seek. The paradox is that you might...        Seth Godin
Categories: thinktime

Rusty Russell: Broadband Speeds, 2 Years Later

Planet Linux Australia - Thu 06th Jul 2017 21:07

Two years ago, considering the blocksize debate, I made two attempts to measure average bandwidth growth, first using Akamai serving numbers (which gave an answer of 17% per year), and then using fixed-line broadband data from OFCOM UK, which gave an answer of 30% per annum.

We have two years more of data since then, so let’s take another look.

OFCOM (UK) Fixed Broadband Data

First, the OFCOM data:

  • Average download speed in November 2008 was 3.6Mbit
  • Average download speed in November 2014 was 22.8Mbit
  • Average download speed in November 2016 was 36.2Mbit
  • Average upload speed in November 2008 to April 2009 was 0.43Mbit/s
  • Average upload speed in November 2014 was 2.9Mbit
  • Average upload speed in November 2016 was 4.3Mbit

So in the last two years, we’ve seen 26% increase in download speed, and 22% increase in upload, bringing us down from 36/37% to 33% over the 8 years. The divergence of download and upload improvements is concerning (I previously assumed they were the same, but we have to design for the lesser of the two for a peer-to-peer system).

The idea that upload speed may be topping out is reflected in the Nov-2016 report, which notes only an 8% upload increase in services advertised as “30Mbit” or above.

Akamai’s State Of The Internet Reports

Now let’s look at Akamai’s Q1 2016 report and Q1-2017 report.

  • Annual global average speed in Q1 2015 – Q1 2016: 23%
  • Annual global average speed in Q1 2016 – Q1 2017: 15%

This gives an estimate of 19% per annum in the last two years. Reassuringly, the US and UK (both fairly high-bandwidth countries, considered in my previous post to be a good estimate for the future of other countries) have increased by 26% and 19% in the last two years, indicating there’s no immediate ceiling to bandwidth.

You can play with the numbers for different geographies on the Akamai site.

Conclusion: 19% Is A Conservative Estimate

17% growth now seems a little pessimistic: in the last 9 years the US Akamai numbers suggest the US has increased by 19% per annum, the UK by almost 21%.  The gloss seems to be coming off the UK fixed-broadband numbers, but they’re still 22% upload increase for the last two years.  Even Australia and the Philippines have managed almost 21%.

Categories: thinktime

Does it help?

Seth Godin - Thu 06th Jul 2017 18:07
Isn't this the essence of design thinking? I have a great wool hat that I wear in the winter. Does it help? Well, that depends on what it's for. If it's designed to keep me warm, then yes, it helps....        Seth Godin
Categories: thinktime

Simple approach to the things you check

Seth Godin - Wed 05th Jul 2017 18:07
The data, the dashboard, the comments, the statuses, the likes, the rankings: Check them half as often and do twice as much with what you learn. Then, after you've gotten good at that, repeat the math: Check them half as...        Seth Godin
Categories: thinktime

Danielle Madeley: python-pkcs11 with the Nitrokey HSM

Planet Linux Australia - Tue 04th Jul 2017 23:07

So my Nitrokey HSM arrived and it works great, thanks to the Nitrokey peeps for sending me one.

Because the OpenSC PKCS #11 module is a little more lightweight than some of the other vendors, which often implement mechanisms that are not actually supported by the hardware (e.g. the Opencryptoki TPM module), I wrote up some documentation on how to use the device, focusing on how to extract the public keys for using outside of PKCS #11, as the Nitrokey doesn’t implement any of the public key functions.

Nitrokey with python-pkcs11

This also encouraged me to add a whole bunch more of the import/extraction functions for the diverse key formats, including getting very frustrated at the lack of documentation for little things like how OpenSSL stores EC public keys (the answer is as SubjectPublicKeyInfo from X.509), although I think there might be some operating system specific glitches with encoding some DER structures. I think I need to move from pyasn1 to asn1crypto.

Categories: thinktime

The cost of independence

Seth Godin - Tue 04th Jul 2017 17:07
Freedom comes with choice and choice comes with responsibility. Why do people willingly give up their freedom to a boss, a method or even a despot? Why do successful entrepreneurs who start a new company take on investors even when...        Seth Godin
Categories: thinktime

Whose business are you minding?

Seth Godin - Mon 03rd Jul 2017 19:07
Industries have rules. Rules and benefits. Hollywood requires agents, casting calls, big budgets and content aimed at a certain part of a certain market. If you follow enough of the rules, the thinking goes, you get a multi-million dollar budget...        Seth Godin
Categories: thinktime

OpenSTEM: The Science of Cats

Planet Linux Australia - Mon 03rd Jul 2017 09:07

Ah, the comfortable cat! Most people agree that cats are experts at being comfortable and getting the best out of life, with the assistance of their human friends – but how did this come about? Geneticists and historians are continuing to study how cats and people came to live together and how cats came to organise themselves into such a good deal in their relationship with humans. Cats are often allowed liberties that few other animals, even domestic animals, can get away with – they are fed and usually pampered with comfortable beds (including human furniture), are kept warm, cuddled on demand; and, very often, are not even asked to provide anything except affection (on their terms!) in return. Often thought of as solitary animals, cats’ social behaviour is actually a lot more complex and recently further insights have been gained about how cats and humans came to enjoy the relationship that they have today.

Many people know that the Ancient Egyptians came to certain agreements with cats – cats are depicted in some of their art and mummified cats have been found. It is believed that cats may have been worshipped as representatives of the Cat Goddess, Bastet – interestingly enough, a goddess of war! Statues of cats from Ancient Egypt emphasise their regal bearing and tendency towards supercilious expressions. Cats were present in Egyptian art by 1950 B.C. and it was long thought that Egyptians were the first to domesticate the cat. However, in 2004 a cat was found buried with a human  on the island of Cyprus in the Mediterranean 9,500 years ago, making it the earliest known cat associated with humans. This date was many thousands of years earlier than Egyptian cats. In 2008 a site in the Nile Valley was found which contained the remains of 6 cats – a male, a female and 4 kittens, which seemed to have been cared for by people about 6,000 years ago.

African Wild Cat, photo by Sonelle, CC-BY-SA

It is now fairly well accepted that cats domesticated people, rather than the other way round! Papers refer to cats as having “self-domesticated”, which sounds in line with cat behaviour. Genetically all modern cats are related to African (also called Near Eastern) wild cats 8,000 years ago. There was an attempt to domesticate leopard cats about 7,500 years ago in China, but none of these animals contributed to the genetic material of the world’s modern cat populations. As humans in the Near East developed agriculture and started to live in settled villages, after 10,000 years ago, cats were attracted to these ready sources of food and more. The steady supply of food from agriculture allowed people to live in permanent villages. Unfortunately, these villages, stocked with food, also attracted other animals, such as rats and mice, not as welcome and potential carriers of disease. The rats and mice were a source of food for the cats who probably arrived in the villages as independent, nocturnal hunters, rather than as deliberately being encouraged by people.

Detail of cat from tomb of Nebamun

Once cats were living in close proximity to people, trust developed and soon cats were helping humans in the hunt, as is shown in this detail from an Egyptian tomb painting on the right. Over time, cats became pets and part of the family and followed farmers from Turkey across into Europe, as well as being painted sitting under dining tables in Egypt. People started to interfere with the breeding of cats and it is now thought that the Egyptians selected more social, rather than more territorial cats. Contrary to the popular belief that cats are innately solitary, in fact African Wild Cats have complex social behaviour, much of which has been inherited by the domestic cat. African wild cats live in loosely affiliated groups made up mostly of female cats who raise kittens together. There are some males associated with the group, but they tend to visit infrequently and have a larger range, visiting several of the groups of females and kittens. The female cats take turns at nursing, looking after the kittens and hunting. The adult females share food only with their own kittens and not with the other adults. Cats recognise who belongs to their group and who doesn’t and tend to be aggressive to those outside the group. Younger cats are more tolerant of strangers, until they form their own groups. Males are not usually social towards each other, but occasionally tolerate each other in loose ‘brotherhoods’.

In our homes we form the social group, which may include one or more cats. If there is more than one cat these may subdivide themselves into cliques or factions. Pairs of cats raised together often remain closely bonded and affectionate for life. Other cats (especially males) may isolate themselves from the group and do not want to interact with other cats. Cats that are happy on their own do not need other cats for company. It is more common to find stressed cats in multi-cat households. Cats will tolerate other cats best if they are introduced when young. After 2 years of age cats are less tolerant of newcomers to the group. Humans take the place of parents in their cats’ lives. Cats who grow up with humans retain some psychological traits from kittenhood and never achieve full psychological maturity.

At the time that humans were learning to manipulate the environment to their own advantage by domesticating plants and animals, cats started learning to manipulate us. They have now managed to achieve very comfortable and prosperous lives with humans and have followed humans around the planet. Cats arrived in Australia with the First Fleet, having found a very comfortable niche on sailing ships helping to control vermin. Matthew Flinders‘ cat, Trim, became famous as a result of the book Flinders wrote about him. However, cats have had a devastating effect on the native wildlife of Australia. They kill millions of native animals every year, possibly even millions each night. It is thought that they have been responsible for the extinction of numbers of native mice and small marsupial species. Cats are very efficient and deadly nocturnal hunters. It is recommended that all cats are kept restrained indoors or in runs, especially at night. We must not forget that our cuddly companions are still carnivorous predators.

Categories: thinktime

Just because you can doesn't mean you should

Seth Godin - Sun 02nd Jul 2017 18:07
It might be a market you can enter, but that doesn't mean it will reward your time and effort. It might be an all-you-can-eat situation, but there's a difference between all you can eat and all you care to eat....        Seth Godin
Categories: thinktime

Your Bob Dylan story

Seth Godin - Sat 01st Jul 2017 18:07
I know dozens of people who have a story about meeting, or nearly meeting, or somehow engaging with Bob Dylan. And just about everyone they know has questions about him, about those encounters, about what it was like. My guess...        Seth Godin
Categories: thinktime

Two confusions

Seth Godin - Fri 30th Jun 2017 18:06
Those things you're bad at? You're not nearly as bad at them as you fear. And those things you're great at? Probably not nearly as good as you hope. We beat ourselves up a lot, but often focus on the...        Seth Godin
Categories: thinktime

Four ways to improve customer service

Seth Godin - Thu 29th Jun 2017 18:06
Delegate it to your customers. Let them give feedback, good and bad, early and often. Delegate it to your managers. Build in close monitoring, training and feedback. Have them walk the floor, co-creating with their teams. Use technology. Monitor digital...        Seth Godin
Categories: thinktime

Creating discomfort

Seth Godin - Wed 28th Jun 2017 18:06
If you're seeking to create positive change in your community, it's almost certain you'll be creating discomfort as well. Want to upgrade the local playground? It sounds like it will be universally embraced by parents and everyone who cares about...        Seth Godin
Categories: thinktime

Team Conflict: Four Ways to Deflate the Discord that&#8217;s Killing Your Team

a list apart - Wed 28th Jun 2017 00:06

It was supposed to be a simple web project. Our client needed a site that would allow users to create, deploy and review survey results. Aside from some APIs that weren’t done, I wasn’t very worried about the project. I was surprised that my product manager was spending so much time at the client’s office.

Then, she explained the problem. It seemed that the leaders of product, UX and engineering didn’t speak to each other and, as a result, she had to walk from office to office getting information and decisions.

When two people have a bad interaction, they can work it out or let the conflict grow, spreading it to other team members and their leaders.

The conflicts probably started small. One bad interaction, then another, then people don’t like each other, then teams don’t work together well. The small scrape becomes a festering wound that slows things down, limits creativity and lowers morale.

Somehow as a kid working my way through school I discovered I had a knack for getting around individuals or groups that were fighting with each other. I simply figured out who I needed to help me accomplish a task, and I learned how to convince, cajole or charm them into doing it. I went on to teach these skills to my teams.

That sufficed for a while. But as I became a department head and an adviser to my clients, I realized it’s not enough to make it work. I needed to learn how to make it better. I needed to find a way to stop the infighting I’ve seen plague organizations my entire career. I needed to put aside my tendency to make the quick fix and have hard conversations.

It’s messy, awkward and hard for team leaders to resolve conflict but the results are absolutely worth it. You don’t need a big training program, a touchy-feely retreat or an expensive consultant. Team members or team leads don’t have to like each other. What they have to do is find common ground, a measure of respect for one another, and a willingness to work together to benefit the project.

Here are four ways to approach the problem.

Start talking

No matter how it looks at first, it’s always a people problem. Gerald M. Weinberg, The Secrets of Consulting: A Guide to Giving and Getting Advice Successfully

Resist the urge to wait for the perfect time to address team conflict. There’s no such thing. There will always be another deadline, another rollout, another challenge to be met.

In our office, a UX designer and product manager were having trouble getting along. Rather than take responsibility, they each blamed our “process” and said we needed to clarify roles and procedures. In other words, they each wanted to be deemed “in charge” of the project. Certainly I could have taken that bullet and begun a full-on assessment of our processes and structure. By taking the blame for a bad company framework, I could have dodged some difficult conversations.  But I knew our process wasn’t the problem.

First, I coached the product manager to be vulnerable, not an easy thing for him to do. I asked him to share his concerns and his desire to have a more productive relationship with the UX designer. The PM’s willingness to be uncomfortable and open about his concerns lifted the tension. Once he acknowledged the elephant in the room—namely that the UX designer was not happy working with him—the designer became more willing to risk being honest. Eventually, they were able to find a solution to their disagreements on the project, largely because they were willing to give each other a measure of respect.

The worst thing I’ve seen is when leaders move people from team to team hoping that they will magically find a group of people that work well together, and work well with them. Sometimes the relocated team members have no idea that their behavior or performance isn’t acceptable. Instead of solving the problem, this just spreads the dissatisfaction.

Instead, be clear right from the beginning that you want teams that will be open about challenges, feel safe discussing conflicts, and be accountable for solving them.

Have a clear purpose

Although many aspects of our collective endeavor are open for discussion, choice of mountain is not among them. J. Richard Hackman, Leading Teams: Setting the Stage for Great Performances

I was working on an enterprise CMS re-design and re-platform. Our weekly review and estimation sessions were some of the most painful meetings of my career. There was no trust or shared purpose—even estimating a simple task was a big fight.

When purpose and priorities are murky you are likely to find conflict. When the team doesn’t know what mountain they are trying to climb, they tend to focus on the parts of the project that are most relevant to them. With each team member jealously guarding his or her little ledge, it’s almost impossible to have cooperation.

This assault on productivity is likely because the project objective is non-existent, or muddled nonsense, or so broad the team doesn’t see how it can have an impact. Or, maybe the objective is a moving target, constantly shifting.

Size can be a factor.  I’ve seen enterprise teams with clear missions and startups with such world-changing objectives they can’t figure out how to ship something that costs less than a million dollars.

When I’m meeting with prospects or new clients I look at three areas to see if they are having this problem:

  • What language do they use to describe each other? Disconnected teams say “UX thinks,” “The dev team” or “product wants.” Unified teams say “we.”
  • How easy or hard is task estimation? Disconnected teams fight about the level of difficulty. United teams talk about tradeoffs and argue about what’s best for the product or customers.
  • Can they easily and consistently describe their purpose? Disconnected teams don’t have a crisp and consistent answer. Unified teams nod their heads when one of their members shares a concise answer.

If a team is disconnected, it’s likely because you haven’t given them a common goal. A single email or a fancy deck isn’t enough. Make your objectives simple and repeat them so much that the team groans every time you start.

Plan conversations

Words do not sit in our brains in isolation. Each word is surrounded by its own connotations, memories and associations Simon Lancaster, Winning Minds: Secrets From the Language of Leadership

Years ago I was frustrated to tears by a manager who, I felt, took from me the product I spent two years building. I knew I needed to talk with him but I struggled to find a productive way to tell him why I was upset.  (Telling someone he is being a jackass is not productive.)

A good friend in HR helped me script the conversation. It had three parts:

  • I really work well when…
  • This situation is bothering me because…
  • What I’d like to see happen is…

Leaders have an important role to play in resolving issues. When a leader decides that their person is right and another person is wrong it turns a team problem into an organization problem. Instead we should should provide perspective, context and show how actions could be misunderstood.

Leaders also need to quickly, clearly and strongly call about bad behavior. When I found out one of my people raised their voice at a colleague, I made it clear that wasn’t acceptable and shouldn’t happen again. He admitted that he lost his cool, apologized and then we started working on the resolving the situation.

Require accountability Being responsible sometimes means pissing people off. General Colin Powell,former U.S. Secretary of State

If you have a problem and you go to Holly Paul, an inspiring HR leader, you can expect that she will listen. You can also expect that she’ll work with you on a plan to resolve it. Most importantly you can expect she will make sure you are doing what you said you’d do when you said you would do it.

Before I met Holly I would listen to problems then try to go solve them. Now I work with the person and tell them that I will be checking back with them, often putting the reminder in my calendar during the conversation so I don’t forget.

Since I started focusing on fixing conflict, I’ve seen great changes on my team. Many of them have started for the first time dealing with the people, fixing their issues and forging much stronger relationships. Our team is stronger and having a greater influence on the organization.

It’s messy, awkward and hard. I’ve been working on this for a long time and I still make mistakes. I still don’t always want to push when I meet resistance. This will never be easy, but it will be worth it and it’s your responsibility as a leader. For however long these people are with you, you need to make them better as individuals and a unit.

You don’t need a big training, a touchy-feely retreat or an expensive consultant. You just need to start doing the work every day. The rest will come.

Categories: thinktime


Subscribe to kattekrab aggregator