We have un-matched partnership networks with world-class tech vendors.
Management and Metadata for Files and Objects
Starfish is a unique and versatile application for managing files and objects at any scale, from departmental file shares to the world's largest supercomputing file systems. Its central insight: at billions of files, the problem isn't storing data, it's knowing what you have.
Starfish builds and maintains a metadata catalogue of massive file systems, combining file system scanning with a rich tagging and reporting engine. Its unique approach to metadata enriches understanding of how file collections relate to the business or science: know what you have, how much, how many, how new, how old and then take action on those discoveries. Actions include tiering cold data to object or tape, archiving completed projects, deleting redundant data, chargeback reporting by lab or grant, and locating candidate datasets. It operates at scales (tens of billions of files, exabytes) where generic tools collapse, and works across parallel file systems (Storage Scale, Lustre), NAS, and object stores.
Research file systems are notoriously opaque: Starfish gives HPC operators the visibility to reclaim expensive tier-1 capacity, enforce data policies, and defer storage purchases which frequently pays for itself by avoiding flash expansion.
AI starts with finding and curating training data; Starfish's catalogue turns institutional data sprawl into a searchable asset inventory, helping teams assemble datasets, track provenance, and manage the lifecycle of model artefacts and training.