Holodeck Data Analytics & Security — Video with Transcript2018-07-24T13:38:30-07:00

Holodeck Data Analytics & Security — Video & Transcript

Shiv Sikand, Founder & EVP, IC Manage (edited transcript – DAC 6.25.18)
See Demo

Automatic Logging for Analytics

Now one of the accidental discoveries of our architecture very early on was that because we were always translating this virtual layer to a real layer, we know every transaction on a file in file namespace. Storage has typically been in block space.

Your IT guys can tell you that you’re getting a very high block rate, but they can’t help you identify a specific or local bottleneck in a workflow. Why is this application suddenly running slow? “Well I’m not really sure.”

But in Holodeck, we know the name of the file, we know the writer, we know the reader, we know every stat, we know every open, we know every close, we know every rename, we know absolutely everything.

And this spews out a massive log of events that get consumed by something like Elasticsearch, and then you can build your own dashboard.

Outliers & Security

You can use it not only for optimizing your workflows and finding out: “hey where am I inefficient?”, “which workflows are broken?”, “which workflows are good?”, “which ones can be optimized?” — and more importantly you can use it to detect outliers.

In this modern science of big data and AI, we’re focused around outlier detection. Why did this guy or this team who are usually working 9 to 5 are now working at 2am? That’s an indicator. Or who is this specific individual who used to only do reads in this one area is now doing massive reads across hundreds of projects.

Data exfiltration — when breaches occur — what tends to happen is that you lose you know tons of data, because you didn’t know about it. You don’t ready “I lost 3 bytes of data”.  You read about people who’ve lost terabytes of data. Their entire corporate databases, you know Equifax or whatever being the last one because they don’t know.

With this outlier detection behavior, you can very, very easily detect what’s going on.  Because we’re working in file space and not in block space.

Chains of Custody

Now chains of custody is a very important issue in semiconductor when you’re using other people’s IP, and you want to make sure that that IP is being used appropriately according to the SLAs you signed for that data.

It’s very simple in IC Manage Holodeck to add a simple change of custody, and we use a distributed ledger in the metadata to track where a file has been.