<

Martin Sumner

Worked long enough in networks, to always blame the application

Martin was the technical lead on one of the largest (by volume) public sector systems in Europe - the UK National Health Service Spine II. The architecture of this was inspired by Erlang, and the Actor Model - and Spine II has maintained 5 nines availability since its go-live in 2014. Spine II is a big user of Riak, and for the past 3 years Martin has worked full-time on developing and maintaining the open-source Riak project - learning to love Erlang and the BEAM in the process.

Past Activities

Martin Sumner
Code BEAM STO 2019
17 May 2019
16.15 - 17.00

Evolution of Riak - discovering and resolving problems in distributed systems

This talk will cover some problems of building a distributed system framework:

  • Mass adoption of scale-out with difficult developer trade-offs is hard to achieve;
  • The problems of testing claims of correctness;
  • Replication of activity and the impact on throughput, and replication of data can itself be a cause of data loss;
  • Non-conformity of performance between nodes is very difficult to manage (one slow node).

How the Riak framework is evolving to solve these problems:

  • Through the power of open source, and the capability to manage with small numbers of contributors spread globally, whilst better leveraging the BEAM eco-system to get progress for "free";
  • Through extended use of property-based testing;
  • Through increased specialisation of sub-components to always do "just enough" work, with better alignment between local storage layout and overall database features;
  • Through reactive work distribution, and minimisation of scheduled background tasks.

OBJECTIVES

An opportunity to talk frankly about the problems that Riak has faced, and raise some optimism about how lessons learnt may give Riak a worthwhile future as an open-source Erlang project.

TARGET AUDIENCE

People interested in the theory distributed databases, the practicalities of real-world scale-out problems and their solutions.

Martin Sumner
Code BEAM STO 2018
01 Jun 2018
13.40 - 14.25

Riak 3.0 and efficient anti-entropy - bringing certainty to eventually

In distributed databases, data will be stored in multiple machines across multiple locations - but when changes could be passed to any of those machines how should one confirm that not only has all data reached all locations, but that the data is still there later?

Work is underway for Riak KV 3.0, the first major release of Riak KV since the project went fully open-source, to try and improve the efficiency of how the entropy problem is addressed. The talk should explain why and how this issue is to be handled, and what feature enhancements this will allow going forward.

The improvements are focused on re-designing Riak’s use of Merkle Trees, a data structure recently popularised though its use in Blockchain systems. The redesign changes how Merkle Trees are stored and calculated, and includes a mechanism for aligning the querying of data in Log-Structured Merge Trees with the Merkle Trees that represent the data.

OBJECTIVES

  • To provide insights into the future features of the open source Erlang database Riak KV.
  • Give an overview of some of the hidden problems in achieving eventual consistency and long-term data durability in distributed systems.
  • Highlight the benefits of optimised Merkle Tree method in comparison other interesting alternatives - for example the server-wide clocks method in DottedDB.