Showing posts from May, 2012

A sweep through my Instapaper for May 2012

Here are some of the presentations/blog posts/articles I read this month, as saved in my Instapaper account. Maybe you'll find something useful in there too.
"Blameless PostMortems and a Just Culture" -- John Allspaw describes the post mortem process at Etsy, in particular how it encourages engineers to own up to their mistakes and to help others avoid them, all in a safe and blame-free environment"'s Continuous Integration" -- good overview of what should be by now industry-standard methods and tools for rapid deployments and continuous integration"Big List of 20 Common Bottlenecks" -- via the super useful High Scalability blog, a list of bottlenecks that you will hit one way or another if you do any serious high-volume systems work"BASE: An ACID alternative" -- Dan Pritchett from EBay coined the term BASE (basically available, soft state, eventually consistent) to compare and contrast NoSQL systems with traditional ACID-based r…

The correct way of using DynamoDB BatchWriteItem with boto

In my previous post I wrote about the advantages of using the BatchWriteItem functionality in DynamoDB. As it turns out, I was overly optimistic when I wrote my initial code: I only called the batch_write_item method of the layer2 module in boto once.

The problem with this approach is that many of the batched inserts can fail, and in practice this happens quite frequently, probably because of transient network errors. The correct approach is to inspect the response object returned by batch_write_item -- here is an example of such an object:

{'Responses': {'mytable': {'ConsumedCapacityUnits': 5.0}},
 'UnprocessedItems': {'mytable': [
{'PutRequest': {'Item': {'mykey': 'key1', 'myvalue': 'value1'}}},
{'PutRequest': {'Item': {'mykey': 'key2', 'myvalue': 'value2'}}},
{'PutRequest': {'Item': {'mykey': 'key3', 'myvalue': &#…