IC Manage PeerCache

IC Manage PeerCache gives engineers massively parallel workflows by accelerating file transfers of multi-terabyte project data – both managed and generated files.

Because of the dramatic increase in the volume of generated data at 28 nm and below, engineers cannot easily make multiple copies: the filer storage cost is too high, and the time to make a physical copies is too long.

PeerCache solves this data volume problem by utilizing compute farm nodes as peer-to-peer caches. It delivers multi-terabyte project data copies in minutes, and drastically reduces storage costs.

Abstract network design with colourful dots connecting to each other

It is a 100% software solution that accelerates NetApp, Isilon, and VMware. It speeds up all your project data, both managed and generated files, and speeds up all DM systems.

IC Manage PeerCache removes the project data serial flow bottleneck that engineering teams are struggling with, when forced to share a very limited number of master copies. It paves the way to massively parallel project data workflows. Engineers are no longer constrained from making any changes while having to wait for long-running jobs and other users in shared directories to finish.

Unique Underlying Technology

IC Manage PeerCache has three key elements to enable massively parallel workflows.

Virtual Workspaces

With Virtual workspaces, the file system interface remains the same, however, the descriptive file metadata is separated from the actual file content. PeerCache allows independent control of the metadata and the file content.  It has a separate transport layer to move the file content bytes around.

Virtual workspaces eliminate the need to make physical copies, reducing both copy time and storage requirements by 10-100x, since there is no duplication of content.

They also offer on-demand access. All files are accessible immediately, with content delivered as needed.

Peer-To-Peer Network

IC Manage PeerCache provides a unique peer-to-peer network which takes advantage of the SSD caches that are increasingly finding their way into these compute farms for use as scratch or local storage. It then uses this peer-to-peer network to accelerate delivery of workspace content.

Traditional serial file data transport mechanisms have limited, on demand access speed (reads), and “write” and metadata “stat” performance.  Even if you add more file storage to the central filer, you are still limited by the speed of a single NFS thread. It still doesn’t give consistent I/O performance across very large data sets, as most shared storage systems suffer from variable latency as they try to scale, slowing application performance.

Semiconductor companies have compute farms with large numbers of nodes. However, while phones and laptops today use very fast local SSD, either as drives or NVMe (non-volatile memory express), compute nodes today can only use SSD as temporary scratch space. The files cannot be shared between nodes because local storage is not networked. Further, these local silos must be manually managed, copied and backed up.

PeerCache’s breakthrough technology creates a peer-to-peer network out of your compute farm, using the compute nodes’ scratch HDD and/or SSD local storage as caches, delivering parallel clones for almost free.

Virtual workspaces plus this peer-to-peer network with local caching gives development teams near-instant content delivery (reads).

Local Read *and* Write Caching

IC Manage PeerCache has a hybrid architecture, which combines the existing NFS storage infrastructure and local SSD performance — all using software only.

So instead of streaming to a central network filer, we write to the local SSD which automatically backs up to the filer.  Thus all “stats”, “reads” and “writes” become local.

Further, the writes are copied back to your existing NFS infrastructure and/or DM system for safe keeping and backup. This maintains the integrity of your data while also making it accessible from any host on the network.

Because the data is written back to the filer, it is fully compatible with your existing infrastructure and best practices.

The local drive serves as a high performance cache for both reads and writes, delivering higher throughput for all of your analysis jobs.

PeerCache supports all major DM systems, both commercial and open source, because it works at the file level, presenting an identical file system interface.

IC Manage PeerCache Advantages

Filer Storage savings

More and more companies are investing in compute node storage to accelerate their application runtime. In a typical scenario, a company might have 800 nodes in a compute farm, each with 500 GB of local cache.

Using ICM PeerCache, the company would have immediate access to an additional 400 TB of available and accelerated networked workspace.

Speed Improvement

Current NFS architecture can deliver between 100 to 500 MB/sec of throughput, while an NVMe SSD compute node can deliver more than 2GB/sec

Thus PeerCache gives you 4-20X the speed potential of existing systems

Massively Parallel Workflows

When engineers are forced into serial flows, they must wait for long tool runs before they can continue their tasks. Further, when they use a shared model for the generated data, all the data management tools they used to have at their disposal are not available, such as isolation branching and intermediate versioning of the combined source and generated data.

So they have the additional problem of manually coordinating changes between users while the jobs are actually running; they can inadvertently end up with incorrect versions and overwritten data.

With parallel workflows, engineers no longer have to wait for someone else to finish with their physical copy.

Any user can now clone any authorized workspace at any moment in time – even a terabyte-sized full chip workspace. The clones occur in near-zero time, and include both managed and generated data. No additional storage is consumed until changes are made.

Further, the clones can now function as branches. Derived data is now “managed” in that you can track and switch between multiple configurations. Distributed version control system (DVCS) tools such as Git or Perforce DVCS can be used for isolated revision control within the clone, which can then be pushed back to the main tree.

Development Team Benefits

Because PeerCache removes the serial development bottleneck, development teams can benefit from the parallel, isolated workflows available in situations such as when they are:

  • Running numerous corners and 100’s of different scenarios, where each sub block may be dozens of GBs.
  • Doing ECO on the top level databases, where the fixes must be communicated to the remote sub block owners to run all their analysis and then create new workspaces for a final top-level check.
  • Executing physical implementation with multiple terabytes of generated files, where designers must wait for long runs to complete before they can make changes for their next set of tasks.