Systems Research

• Current Projects
• Past Projects
• Contact Us

Related Links

• Research Home
• Get an Account
• Networking Research
• OSC Networking

Research Reports

• 2009 Research Highlights
• 2008 Research Report
• 2007 Research Report

Applicability of Object-Based Storage Devices in Parallel File Systems

Administrivia

Current Personnel

• Ananth Devulapalli, Principal Investigator
• Iyyappa T. Murugandi, Graduate student
• Da Xu, under-graduate student

Project Alumni

• Pete Wyckoff
• Dennis Dalessandro
• Nawab Ali
• Praveen Ramalingam
• Marcus Hutton

Project summary

While continued improvements in processing speeds and disk densities improve computing over time, the most fundamental advances come from changing the ways in which components interact. Delegating responsibility for some operations from the host processor to intelligent peripherals can improve application performance. Traditional storage technology is based on simple fixed-size accesses with little assistance from disk drives, but an emerging standard for object-based storage devices (OSDs) is being adopted. These devices will offer improvements in performance, scalability and management, and are expected to be available as commodity items soon.

When assembled as a parallel file system, for use in high-performance computing, object-based storage devices offer the potential to improve scalability and throughput by permitting clients to securely and directly access storage. However, while the feature set offered by OSD is richer than that of traditional block-based devices, it does not provide all the functionality needed by a parallel file system.

We will examine multiple aspects of the mismatch between the needs of a parallel file system, in particular PVFS2, and the capabilities of OSD. Topic areas include mapping data to objects, metadata, transport, caching and reliability. Trade-offs arise from the mapping of files to objects, and how to stripe files across multiple objects and disks, in order to obtain good performance. A distributed file system needs to track metadata that describes and connects data. OSDs offer automatic management of some critical metadata components that can be used by the file system. There are transport issues related to flow control and multicast operations that must be solved. Implementing client caching schemes and maintaining data consistency also requires proper application of OSD capabilities.

Our work will examine the feasibility of OSDs for use in parallel file systems, discovering techniques to accommodate this high performance usage model. We will also suggest extensions to the current OSD standard as needed.

Milestones achieved

• Design and implementation of OSD target. It implements almost all mandatory commands in OSD2v3 specification and some interesting optional commands like multi-object commands.
• A light-weight OSD initiator library to enable clients to speak with the OSD target.
• Extensions to enable usage over RDMA Networks also known as iSER.
• An intelligent OSD capable of executing offloaded commands and related infrastructure.

Project deliverables

Peer-reviewed publications

• Nawab Ali, Ananth Devulapalli, Dennis Dalessandro, Pete Wyckoff and P. Sadayappan: Revisiting the Metadata Architecture of Parallel File Systems, 3rd International Petascale Data Storage Workshop (PDSW) [paper | talk]
• Ananth Devulapalli, Dennis Dalessandro, Pete Wyckoff: Data Structure Consistency Using Atomic Operations in Storage Devices, The 5th International Workshop on Storage Network Architecture and Parallel I/Os (SNAPI) 2008. [paper | talk]
• Nawab Ali, Ananth Devulapalli, Dennis Dalessandro, Pete Wyckoff and P. Sadayappan: An OSD-based Approach to Managing Directory Operations in Parallel File Systems, IEEE Cluster 2008. [paper | talk]
• Dennis Dalessandro, Ananth Devulapalli and Pete Wyckoff: Non-Contiguous I/O Support for Object-Based Storage, International Workshop on Parallel Programming Models and Systems Software for High-End Computing (P2S2) 2008 [paper | talk]
• Ananth Devulapalli, Dennis Dalessandro, Pete Wyckoff, Nawab Ali and P. Sadayappan: Integrating Parallel File Systems with Object-Based Storage Devices, Super Computing (SC) 2007 [paper | talk]
• Ananth Devulapalli, Dennis Dalessandro, Pete Wyckoff and Nawab Ali: Attribute Storage Design for Object-based Storage Devices , 24th IEEE Conference on Mass Storage Systems and Technologies (MSST) 2007 [paper | talk | poster]
  
• Dennis Dalessandro, Ananth Devulapalli and Pete Wyckoff: iSER Storage Target for Object-based Storage Devices, The 4th International Workshop on Storage Network Architecture and Parallel I/Os (SNAPI) [paper | talk]

Other publications

• Ananth Devulapalli, Iyyappa T. Murugandi, Da Xu and Pete Wyckoff: Design of an Intelligent Object-based Storage Device, The Ohio Supercomputer Center technical report [techreport]
• Ananth Devulapalli, Dennis Dalessandro, Pete Wyckoff and Nawab Ali: Attribute Storage Design for Object-based Storage Devices , The Ohio Supercomputer Center technical report [techreport]

Software

The software infrastructure is made up of the following elements:

• OSD target and initiator for OSD2v3 spec
• iSER module to enable OSD communications over RDMA capable interconnects like InfiniBand
• Intelligent OSD (iOSD) target and initiator capable of executing offloaded computation.