In order to provide stable and different operations for the GLASS system, several data structures have been used to define objects, The very first data structure is book-list, in this structure there are three main fields, the first is information that can be queried and got from book object description table, the second is information that can be queried and got from gird nodes description table, and the third is information that can be queried and got from volume object description table.
To describe the grid nodes, we use node-list structure. The node-list structure has fields to store the total number of available nodes, and a list of recorder for each of the grid node. The library storage can be composed of several volumes and there can be several grid nodes in each volume. When a book is copied into the library storage system, the book is copied according to the volume attribute of the book and, there can be several nodes, where book will be copied into. And in each of these nodes, some file systems mounted.
The first function book-list-init (book-list structure) does a joint query to several tables in the database and, gets as many attributes as the metadata can provide. The second function book-list-print is for printing the book-list structure.
The function node-list-init first queries the metadata database to get the nodes information belongs to this volume. And then function node-list-init will try to query the node object description table to get some detailed information about each node, such as node IP address and will send a LDAP query to MDS server of each grid node, parse the query result and get detailed nodes information, such as, file system name, free space, then we will further recognize the file systems that can be used for book storage.
In GLASS system, there are many operations, which will talk to an object that can be in a different abstraction level. In general, the library is an object, which has different user and user groups, in which they have different permission to do actions on it. The library is composed by volumes, in each volume there can be several nodes, every node has several file systems can be used.
The operation glass-update is an operation, which copy a new book to a proper file system or update an old book that is already in the system. In this operation, volume attribute can be got from book-list, then by using volume attribute, node-list can be initialized and we can get the proper file system for storing the book. The book will be tarred and transferred to remote node, if wanted as a backup or for storage and then untarred. The procedure of tarring, transferring and untarring are staged to the grid system as a job, so the command can be send from a server and actual operation can take place between other two remote nodes. After the work has done, book information will insert into the table of book object description. Here for the time being, the scheduling policy of glass-update is "largest capacity first", which means we always choose the server that has largest storage capacity. The operation glass-cp is also a low lever operation, which just simply copies a book between two remote nodes The realization of this function is a must simple version of function glass-update. The operation god-cp is also used for copying a book between different working areas.
The operation glass-rm is for remove a book in a volume. In this function, we first find in which node the book is located, and then move the books to a temporary delete directory, and update the book status as deleted in book description table. All of the procedures are submitted as remote jobs to grid system and take place in remote nodes.
The function glass-rc recovers a removed book in a remote node. This function does reverse operation as the function god-rm.
The function func-job-submit and url-copy provide the programming interface to the globus fundamental service, the url-copy provides two different interfaces for remote file transfer and copy. The fun-job-submit receives globus personal gatekeeper port and login name and job description. In our prototype, all the operation that takes place on remote server are customized and submitted from local server.
The operation glass-shutdown is a function for physically take a node out off the system. When a node is to be shutdown and take out off the system, all the books located in this node need to be reloaded to other nodes, to guarantee the data integration. There are two cases, one is that when we maintain mirroring concept and we don't. In the first situation, books in this node can be copied from the mirror volume, so the node can be shutdown at once. In the second case we need firstly to redistribute all books in this node to other nodes in the same volume. The realization of glass-shutdown utilizes the glass-update and glass-cp basic operations.
experiment & results
The work has not evolved into implementation phase and processing optimization, we present some benchmarks with process phases. These benchmarks are useful to verify Grid applicability. Below are the some benchmarks, where GLASS performance depends upon; i) Book Size (Kbs), ii) Free Space Query, iii) Time taken for Tarring and Untarring, iv) Uploading time, v) Retrieval time.
Theoretically, the proposed architecture for storing large bytes of data across the network and access it using grid technology is efficient for storing large amounts of data using Grid and its commodity components which acts as key components and assures a very good price and performance compromise. In this paper, we have discussed the architecture pertaining to the discovery and dissemination of data (multimedia) in an institution or organization, when the resource management is to be considered. Grid is currently being built on a concerted effort among many institutions. Currently we are in first phase of system implementation and designing user interfaces, and have to check, how to enhance it.
- Kubiatowiuz J, Bindel D, Chen Y, Czerwinski S, Eatson P Gunmandi R, "OceanStore: An Architecture for Global - Scale Persistent Storage. In Proceedings of the 9th International Conference on Architectural Support for Programming Language and OS, 2000.
- Foster I, "The anatomy of the Grid: Enabling scalable virtual organizations", Lecture notes in Computer Science, 2001.
- Foster I, Kesselman C, Nick J and Tuecke, S, "The physiology of the grid: open grid services architecture for distributed system integration, 2002.
- Fitzgerald S, Foster I, Kesselman C, von Laszewski G. Smith W and Tuecke S " A Directory Service for Configuring High Performance Distributed Computations." In Proceedings of the 6th IEEE Symposium on High Performance Distributed Computing, pages 365 - 375, 1997,