This morning we had a yoga session at 8am, followed by the welcome-to-week-two pep talk at 10am. It was stressed that we should make the most of this week for building an MVP for our projects.

Found a unicorn in the office #insightfellows #motivationmonday

A photo posted by Katy Chuang, PhD ( on

Indeed, I found a unicorn and felt it was inspirational.

Onto more serious topics… today we all worked more on our server setups. Branching out from the Hadoop/Kafka basic setup.

Caught some discussion about setups briefly - Distributed computing systems are not able to provide all three of the following guarantees:

  1. Consistency;
  2. Availability; and
  3. Partition tolerance

Consistency and availability are mutually exclusive. Availability being great for a website like because they want to have users’ shopping cart actions always available. The order of products added isn’t as important as say chat message order, where consistency is way more important - more so than a minor lag.

- Link forthcoming

All day I switched between working on server setups and then looking at potential data. eBay API would have resale price data but I felt that it wasn’t as interesting as looking at instagram posts so I checked out the latter.

Summary of instagram findings are shown on the next blog post.

2 Link forthcoming