Today is InDay at LinkedIn. We are encouraged to spend one day each month learning, inspiring others, and sharing knowledge with the community. I’ve been at LinkedIn for four months now, and this day gives me the perfect excuse to write a post about two things I love about this place: open source code and big data.
- Apache Kafka: a distributed publish-subscribe messaging system. The design document is a great read, I believe anyone working with distributed systems should check it out. [btw, congratulations Jay on the new baby!]
- Krati: a simple persistent data store with very low latency and high throughput. It’s similar to BDB but written natively in Java. It’s hash-based, which gives it better random access performance than BDB’s BTree structure. Jingwei Wu is one of the best systems programmers I’ve met, and I’m so happy that he’s on our team.
- Project Voldemort: a Java implementation of the Amazon Dynamo paper. It’s some of the cleanest code I’ve seen, and a great place to start if you want to learn about distributed key-value stores. Also, you should follow Alex Feinberg on Twitter and Quora.
- IndexTank, of course