The California Technology Agency is seeking to adopt best practices to make efficient use of the state’s IT data storage infrastructure. It is estimated that the State of California has purchased over six petabytes of data storage over the past five years. In size comparison, it’s been noted that the website YouTube now streams about six petabytes of video every month.

Recognizing data storage as a key infrastructure component, the Technology Agency formed a data storage team as part of the Infrastructure Consolidation Program to review data storage industry best practices, survey the data storage marketplace and determine future state data storage requirements. This research included several state data centers, meetings with 17 data storage companies and interviews of storage experts.

Our team has concluded the following are key technologies that will align the state with efficient use of energy within our future data storage environments:

Thin Provisioning: With traditional server-storage configurations, servers are allocated storage capacity based on the projected requirements of the application environments they support. This typically results in an over allocation of storage. Thin Provisioning trims the “fat” by eliminating instances of inaccessible capacity and improves storage utilization rates. This can result in an increase utilization of storage space from 30-40% to the 75%-to-80% range.

Data Deduplication: This technology eliminates redundant data at a low level, which in- turn improves overall storage efficiency. Properly implemented, using data de-duplication technology can reduce storage requirements by up to 20:1.

Tiered Storage: Using different classes of storage and data software migration tools allows storage administrators to assign storage to the value of the data and the storage medium to which it resides. By assigning less accessed data to slower speed storage, California agencies will be able to reduce overall energy consumption and make less frequently accessed data available at a lower cost.

Consolidation continues on its goals including reducing energy, reducing our data center square footage, and moving mission-critical applications into key data centers. The Infrastructure Consolidation Program is working together with many individuals across the state to accomplish its mission.