Big Data Strategies

How can we handle Big Data, its volume and its velocity?

Big Data has some basic properties:

• It's raw (you can answer more questions from raw data);
• It's not cleaned (don't want to lose any data, even if we think it is not useful); and
• It (should be) immutable--after all, data corruption is the cause of most problems in a database.

Timestamps and Small Tables

Instead of this:

We would have this:

So now changing any of the values for any of the ID's is substantially easier.

If we want to add more data to one of the ID's, we can do that.
If we want to remove a field from one of the people, we can just remove the row for e.g. "town".

Add and Never Delete

Your data sets will get bigger. But never delete a fact! Even if it is outdated. You can construct a state of affairs at any point by keeping a history.

Instead of this:

Do this:

That way, it is much easier to check how many friends Jose had at timestamp 1449051881. You could go further, and save which friend was added at what time.