The ITC Distributed File System: Principles and Design

This work describes the development of a flexible and adaptable distributed file system model where the main concepts of distributed computing are intrinsically incor-porated. The file system incorporates characteristics such as transparency, scalability, fault-tolerance, cryptography, support for low-cost hardware, easy configuration and file manipulation.

ACM Transactions on Computer Systems, 1988

The Andrew File System is a location-transparent distributed tile system that will eventually span more than 5000 workstations at Carnegie Mellon University. Large scale affects performance and complicates system operation. In this paper we present observations of a prototype implementation, motivate changes in the areas of cache validation, server process structure, name translation, and low-level storage representation, and quantitatively demonstrate Andrew's ability to scale gracefully. We establish the importance of whole-file transfer and caching in Andrew by comparing its performance with that of Sun Microsystem's NFS tile system. We also show how the aggregation of files into volumes improves the operability of the system.

Proceedings of the 2nd ACM SIGCOMM workshop on Home networks - HomeNets '11, 2011

Devices forming a Home Network have different capabilities and interfaces, discouraging users to organize their large digital content libraries. To help users, we propose to organize the Home Network according to a gateway-centric architecture, where the content access unification is realized at the file system level and where no additional software installation on devices is required. Solutions for realizing this unification individually exist for the various devices making up the Home Network (UPnP/DLNA devices, personal computers, cloud storage systems, etc). Unifying the content access at the file system level offers a powerful lever for many legacy applications, as far as these applications can access all shared data in the Home Network. Users can thus continue to use their PC's file manager or favorite media player to browse or display shared content. An indexing application, running on the gateway, possibly managed by the ISP and accessible from any device via a simple web interface, enables more powerful content retrieval and user experience. Such application may be enriched to offer additional services like content format adaptation, duplication detection or automatic backup. Lastly we describe how this gateway-centric architecture can be leveraged by cloud applications such as distributed storage systems.

Proceedings of the 8th …, 2007

The global inter-networking infrastructure that has become essential for contemporary day-to-day computing and communication tasks, has also enabled the deployment of several large-scale data sharing overlays. Communities collaboratively aggregate and distribute file and storage resources either in the controlled environment of the Grid, or hidden under the anonymity cloak created by peer-to-peer protocols. Both designs exhibit unique properties and characteristics: Peer-to-peer algorithms address the formation of vast, heterogeneous and dynamic sharing networks, while Grids focus on policy enforcement and accounting features. A distributed data management facility that will assimilate respective practices has been envisioned by numerous related research initiatives, especially when there is a need to incorporate disperse resources in large pools, without relinquishing participants of their respective rights. In this paper, we describe the Distributed File Services (DFS) architecture -a peer-to-peer service overlay, which allows distinct administrative entities to form arbitrary file distribution relationships. Each DFS peer can be uniquely authenticated and maintains direct control of its own namespace and storage assets by defining corresponding authorization directives and policies. The peerto-peer nature of the system allows for scalable deployment and resource allocation, either in a stand-alone scenario or in the Grid context. Moreover, we introduce the notion of a "web of files", as a non-hierarchical, global-scale namespace of distributed data collections and elaborate on a prototype implementation that features novel semantics for integrating our architectural principles and concepts into the operating system level.

Peer-to-peer storage systems organize a collection of symmetric nodes to store data without the use of centralized data structures or operations. As a result, they can scale to many thousands of nodes and can be formed directly by clients. This paper describes the design and implementation Mammoth, which implements a traditional UNIX-like hierarchical file system in a peer-to-peer fashion. Each Mammoth node stores a potentially arbitrary collection of directories and files that may be replicated on multiple nodes. The only thing that links these nodes together is that each metadata object encodes the network addresses of the nodes that store it. Data is replicated by a background process whose operation is simplified by the fact that files are stored as journals of immutable versions. An optimistic replication technique is used to allow nodes to read and write whatever version of data they can access, while also ensuring consistency when nodes are connected. In the event of temporary failure, eventual consistency is achieved by ensuring that every replica of a directory or file metadata object receives all updates to the object, irrespective of delivery order. While an update is being propagated every node that receives it cooperates to ensure that the update is delivered, even if the original sender fails. Our prototype is implemented as a user-level NFS server. Its performance is comparable to a standard NFS server and it will be publicly available soon.

2009

The need and use of large scale distributed storage has rapidly increased in last few years. Organizations need Terabytes of storage for their operational data and backups. Large storage systems are the ultimate solution, but they are very expensive and require higher degree of skills for their operation and maintenance e.g. Storage Area Network (SAN). We propose "A Highly Scalable and Efficient Distributed File Storage System" that is reliable, inexpensive and easy to maintain. Our system is based on peer-to-peer network architecture. To ensure the reliability of the system we use a technique of erasure codes known as Luby Transform (LT). The System is designed for deployment in Local Area Networks (LAN) but with minimal changes it can be extended for Wide Area Networks (WAN) and Internet.

2020

1-3Department of CSE, Ramaiah Institute of Technology, Bengaluru, India -------------------------------------------------------------------------***---------------------------------------------------------------------AbstractDistributed system plays an important role in analysis and processing of huge amount of data. It is used for large number of applications such as forecasting, real time data analysis, and various other scientific application. Distributed File Systems has emerged as one of the most effective and convenient solutions to the issue of handling computations with increase in the quantity of data. Distributed file system (DFS) is mainly aimed to enable clients to use a common file system which paves the way for sharing data and resources over physically distributed machines. Many distributed computing projects have tried to implement and design distributed file systems with distinct architectures and have provided with various functionalities with them. This paper trie...

2000

In a distributed file system built on network attached storage, client computers access data directly from shared storage, rather than submitting I/O requests through a server. Without a server marshaling access to data, if a computer fails or becomes isolated in a network partition while holding locks on cached data objects, those objects become inaccessible to other computers until a locking authority can guarantee that the lock holder will not again directly access these data. We describe a server that acts as the locking authority and implements a lease-based protocol for revoking access to data objects locked by an isolated or failed computer. When a lease expires, the server can be assured that the client no longer acts on locked data, and can safely redistribute locks to other clients. During normal operation, this protocol invokes no message overhead, and uses no memory and performs no computation at the locking authority.

1994

This paper proposes the use of a wide-area file system for storing and retrieving documents.We demonstrate that most of the functionality of the World-Wide Web (WWW)information service can be provided by storing documents in AFS. The approach addressesseveral performance problems experienced by WWW servers and clients, suchas increased server and network load, network latency and inadequate security. In addition,the mechanism

Concurrency and Computation: Practice and Experience, 2007

The rising demand for mobile computing has created a need for an improved file system that supports mobile clients. Current file systems with support for mobility provide availability through file replicas that are cached at the client side. However, mobile clients may experience different obstacles with regards to the local cache, such as the limited network bandwidth, the intermittent connection, and serious conflicts when synchronizing back to the server. In this paper, we propose a novel mobile distributed file system design, which provides high availability and reliable storage for files and guarantees that file operations are executed regardless of concurrency and failure issues. The design is intended to fit mobile clients (e.g. PDAs and cell phones) that have limited storage space and cannot store all of the data they need, and yet require access to these data at all times. We adopt a server-side caching in order to guarantee sufficient caching space to all mobile clients, and to ensure the availability of files in the case of clients' failures. We present our algorithm, describe its implementation, simulate its high availability functions, and report on its performance evaluation using a cluster of workstations. Our simulation results indicate clearly that our algorithm exhibits a significant degree of automation and conflict-free mobile file system.