University of California

History and Goals of the UC Shared Research Cluster Services

The Shared Research Cluster Services (ShaRCS) project at the University of California was developed by the UC Office of the President (UCOP), in conjunction with Lawrence Berkeley National Labs (LBNL) and the San Diego Supercomputer Center (SDSC). Initial plans started in 2009, and a survey was taken of the 10 UC campuses to gauge interest in the availability of a dedicated, university-owned, high-performance research computing environment.

Benefits of this system include:

  • privileged access to specialized services and systems designed for the type of research computing needs already in progress on the campuses
  • being a significant attraction to potential new researchers being recruited to UC
  • providing the best available computing technology for researchers to develop and enhance scientific software tools, gaining recognition and exposure for their projects and the university

Having shared cluster computing available within the UC community would:

  • provide greater accessibility for HPC services than is typically available at non-dedicated facilities
  • allow researchers to influence the design of the computing environment, and have greater input and control of architecture and policy
  • allow costs to be shared by the UC as part of the general budget rather than being borne directly by projects
  • foster collaboration among researchers at all UC campuses

The survey of UC campuses resulted in 24 interested parties. These groups represent nine of the UC campuses (all but Merced) and LBNL. The resulting ShaRCS high-productivity compute cluster reflects much of the input given by those respondents, and together they comprise the pilot project group whose work will guide, drive, and test the new systems. Once the pilot phase has completed and the systems have been fine-tuned for campus research projects, it is the goal of the project to offer similar services to an expanded group of projects from the broader UC research community including all 10 campuses and affiliates.

This Web site is hosted by UCOP as part of a group of projects designed to reduce costs through economies of scale in the UC system. You can find out more about those related projects on the Pilot Project page. The ShaRCS design group is headed by UCLA Associate Vice Chancellor Information Technology & CIO Jim Davis, LBNL Deputy CIO Tammy Welcome, UCSD Vice Chancellor of Research Arthur Ellis, UCLA's Executive Liaison Warren Mori, and UCOP Associate Vice President and CIO David Ernst. The project is managed by Ramon Lim of UCOP. A complete list is available on the Oversight Board page.

The initial design supports two clusters, one at LBNL for the northern sites, known as ShaRCS North or Mako, and one at SDSC, known as ShaRCS South, or Thresher. The clusters are practically identical, with the same hardware and physical characteristics and only minor, incidental software system variations. The long-term goal is to provide complete transparency between users of the two clusters, so that jobs may run in either environment with no changes to codes. For details of the hardware, see the Hardware Descriptions page. For details of the programming and coding environment, see the Software page.

During the pilot phase, 13 projects will utilize the North Cluster and 11 will utilize the South.

Campuses and organizations designated to run on Mako (North) include:

  • UC Davis
  • UC San Francisco
  • UC Berkeley
  • UC Santa Cruz
  • Berkeley Labs

Campuses designated for initial use of Thresher (South) include:

  • UC Los Angeles
  • UC Santa Barbara
  • UC Riverside
  • UC Irvine
  • UC San Diego

The pilot projects from these campuses and organizations will assist ShaRCS designers and administrators with configuration and management of the clusters. The end goal of the pilot project phase is an ShaRCS environment that is easier and friendlier for researchers to use, allowing them to focus on the science involved in those projects rather than the technicalities of the computing system. Eventually, users will be able to submit jobs to either cluster regardless of their campus affiliation, leveraging the availability of compute cycles at both clusters during high-demand periods.

The initial deployment includes 272 nodes at each location. This include four login nodes and 268 compute nodes on the North Cluster, and 272 compute nodes on the South Cluster, which has an additional login node. Each cluster provides 200+ terabytes of local storage in three differently managed file systems. A support staff will be available during normal business hours to provide help to all projects and users, provided by the support staff of LBNL. The North Cluster is located in the UC Berkeley Data Center, while the South Cluster is located in the SDSC Data Center on the UC San Diego campus. Both centers have a long history of delivering state-of-the-art high-productivity compute facilities, and the ShaRCS resources will continue that tradition to the benefit of all UC organizations.