PeerCache Holodeck Video & Transcript2018-07-25T17:30:08+00:00

PeerCache Holodeck 1&2 — Video with Transcript

Shiv Sikand, Founder & EVP, IC Manage (edited transcript – DAC 6.25.18)
Get a Demo

FULL TRANSCRIPT:  Large, Complex Workflows

Shiv Sikand: A workflow typically has evolved in your company over a long period of time. You have a very large numbers of vendor binaries just to start with. You may have 8 to 10 different versions of every tool that’s installed, and the configuration files specify somewhere in your workflow which release of everything that I’m going to work with.

Often, it’s a very complex mix. It’s not a homogenous environment. On top of that, we have our reference data and our PDKs; some of these PDKs can easily span a terabyte. You could have 10 years’ worth of them.

So, there’s a lot of this data and you don’t really know which bits of design data are referencing which bits of PDK, and which of the vendor binaries that are being run on that flow.

Often, it’s a very complex mix. It’s not a homogenous environment. On top of that, we have our reference data and our PDKs; some of these PDKs can easily span a terabyte. You could have 10 years’ worth of them.

So, there’s a lot of this data and you don’t really know which bits of design data are referencing which bits of PDK, and which of the vendor binaries that are being run on that flow.

IC Manage PeerCache Holodeck

So, what [IC Manage introduces] — and apologies to any “Trekkies” out here — we introduce the Holodeck. And what the Holodeck is, is the ability to project virtually your on-premise environment into the cloud.

You have these very deep file trees and configuration environments that are on-prem, and when you type “ls”, you’ve got the whole world. You see all your different mount points, and you see everything.

Often, it’s a very complex mix. It’s not a homogenous environment. On top of that, we have our reference data and our PDKs; some of these PDKs can easily span a terabyte. You could have 10 years’ worth of them.

So, there’s a lot of this data and you don’t really know which bits of design data are referencing which bits of PDK, and which of the vendor binaries that are being run on that flow.

You have these very deep file trees and configuration environments that are on-prem, and when you type “ls”, you’ve got the whole world. You see all your different mount points, and you see everything.

And what we allow you to do is just take the metadata representation of that world. It’s extremely lightweight. That metadata representation then allows us to on-demand based on the application’s need, transfer just the files — and not even the files, the bits inside the files — into the cloud.

For example, if we had a large, let’s say a 10-gigabyte file, and we only read the first 10 megabytes and then appended 100 megabytes on the end of it, we would only transfer the little bit, and then write back the bit that you wrote onto it.

So, the key to the PeerCache Holodeck is that it’s essentially indistinguishable from your existing on-premise workflow.

PeerCache Cloud Bursting Holodeck – Part 2

Minimizing Data Download

Now, we talked about getting data into the cloud, but obviously getting data out of the cloud is very expensive. All the cloud vendors are the same $10/Gigabyte. That can rack up really, really quickly.

So, if you’re trying to maintain coherency between your on-premise and your cloud, you have a lot of challenges there, because you want to make sure that what you push down, first, is authorized to be pushed down, as you don’t want big SPICE transients going over the wire, but you may want your logs and your results files.

You want that control, and more importantly, our selective write back lets you get that level of control.  But most importantly, all writes are saved as deltas, so we know if you appended to a file, we’ll just send the appended bits back. If you just modified a file somewhere in between, we’ll just send those bits back.

We don’t copy back any data that we read over the wire, because we already know what it is, and we know our writes and we can send those writes back as deltas. So, we can keep your costs under control.

Minimizing Cloud Storage & Attack Surface

However, one of the big challenges is that people worry about attack surfaces.  And if you leave complex file systems in the cloud — I’m not saying they’re unsecure – it’s just a large active surface.

But if you just use ephemeral caching like we do, when the job is done, the data is gone — it doesn’t exist, you were never there. And that’s the power.

PeerCache Case Study Data

Let’s look at some of the results we’ve achieved. This was a workflow and we’re able to extract just the job that needs to run.  These numbers are fairly typical — just the bits that you need.  This is your workflow, but I just want that job. I want to extract that dependency.

You can burst like crazy because the files you’re pulling over are a fraction of what’s actually there on your trees. And remember we can do it at the extent basis, so if you only read, as I mentioned, earlier a few bytes of a file, we’ll only transfer those bytes, we won’t transfer the whole file. We have extent-level granularity on individual files as well.

And because we’re using NVME caching, we can eliminate those I/O wait states presented by your filers. Not only that, we can provide very, very low latency I/O behavior and not variable latency behavior that we see in complex filesystems.

AUDIENCE Q&A

Q1: Is this solution for on-prem or just cloud only?

Shiv Sikand: I’m glad you asked that question. [IC Manage PeerCache] is absolutely suitable for both on-prem and for cloud and we have customers using it in both flavors. You simply need to add NVME to your on-prem nodes and you can run hyper-converged and get massive acceleration on your on-premise as well.

The advantage of course — and why the cloud is particularly attractive right now — is that the cloud is already fully NVME capable. So, your standard big compute instances, for example C5Ds on Amazon, have from 2 to 8 NVMEs already on site. But yes, to answer your specific question it’s absolutely beautiful for scaling on-prem as well.

Q2: For your solution, do I need a license in that EC2 or a specific type of EC2 instance?

Shiv Sikand: We have a number of recommended instances based on your workload. Those go into autoscale groups, and you just run whatever instances you want, and you pay us some fixed amount based on those instances.

Dean Drako: And the reason we primarily are talking about cloud and cloud bursting here in this presentation is because we’re in the infrastructure alley and that’s the topic of the day. But the initial deployments that we did with PeerCache were on-premise for customers who wanted to accelerate their on-prem synthesis and simulation.

Q3: Does it work only on IC Manage controlled repositories or…?

Shiv Sikand: No, it doesn’t care about IC Manage’s repositories — it’s got nothing to do with IC Manage repositories.
It is totally agnostic as to where your files are, it works on any domain, anything that uses files which is every single computer system in the world.