Personal tools
You are here: Home Blog XtreemOS

XtreemFS

2010-09-30

Watch our XtreemOS promotional video!

Filed Under:

We have prepared a video to introduce to the capabilities of XtreemOS:

Take a look at it on the homepage of our project website

Enjoy ;-)

New XtreemOS demo video

We are pleased to announce that we have another new demo video on-line:

the  MaxDB replayer running on XtreemFS (by SAP)

 

2010-02-12

XtreemFS update 1.2.1

Filed Under:
XtreemFS team has just released an update for XtreemFS (version 1.2.1). This version contains mainly bug fixes, e.g. for FreeBSD and Fedora 12, and enhanced replica management. The new scrubber will automatically replace failed replicas.

Source code and packages are available for download on http://www.xtreemfs.org.

There is no change in the database or OSD storage, so an update from 1.2 should work out of the box. The usual advice: backup important data before upgrading.

2009-12-14

Luscious Lebkuchen: XtreemFS 1.2 now available

Filed Under:

XtreemFS 1.2 ("Luscious Lebkuchen") is now available at http://www.xtreemfs.org.

New features:
  • asynchronous MRC database backups (database dumps are created in the background)
  • new GRID SSL mode: SSL is used for authentication only, data is transferred via plain TCP for a better performance
  • Vivaldi-based replica selection and placement
  • MRC and OSD wait and retry for a configurable time span if they cannot reach the DIR on startup
  • faster file copying of files with 'cp' (by providing correct values for 'st_blksize' with 'getattr')
  • additional binaries with standard Unix names (mount.xtreemfs, mkfs.xtreemfs, lsfs.xtreemfs, rmfs.xtreemfs)
  • experimental Hadoop file system driver for XtreemFS
  • web-based admin console
We also fixed most of the bugs recorded in the tracker.

 

When upgrading your XtreemFS installation from 1.1 to 1.2, MRC and DIR databases should be converted automatically, without the need for human intervention. However, for safety reasons, we suggest to create an XML dump with the old release before updating; see "Updating" section in the XtreemFS FAQs (http://www.xtreemfs.org/faq.php).

2009-11-04

Get acquainted with one XtreemOS partner: Barcelona Supercomputing Center (BSC)

 Early in 2004 the Ministry of Education and Science (Spanish Government), Generalitat de Catalunya (local Catalan Government) and Technical University of Catalonia (UPC) took the initiative of creating a National Supercomputing Center in Barcelona. BSC-CNS (Barcelona Supercomputing Center – Centro Nacional de Supercomputaci√≥n) is the National Supercomputing Facility in Spain and was officially constituted in April 2005. BSC-CNS manages MareNostrum, one of the most powerful supercomputers in Europe, located at the Torre Girona chapel. The mission of BSC-CNS is to investigate, develop and manage information technology in order to facilitate scientific progress. With this aim, special dedication has been taken to areas such as Computational Sciences, Life Sciences and Earth Sciences.

All these activities are complementary to each other and very tightly related. In this way, a multidisciplinary loop is set up: our exposure to industrial and non-computer science academic practices improves our understanding of the needs and helps us focusing our basic research towards improving those practices. The result is very positive both for our research work as well as for improving the way we service our society.

 

XtreemOS overall architecture

 

BSC contribution to XtreemOS

 

  • Application execution management

The main task of BSC in XtreemOS is to lead the research and development of the Application Execution Management (AEM) module. This module is in charge of setting the whole environment to run jobs in the grid (build reservations, allocate resources, schedule jobs, launch processes, etc.) and to control and monitor them (maintaining the state, allow thread-level monitoring, etc.).

 

Specifically BSC is highly involved in the scalable job management avoiding any kind of bottleneck as well as on the monitoring that allows users to monitor a job just as if it were a Linux process (you can even monitor the change of thread states). 

  • XtreemFS

 In the data management module, the storage system research group is in charge of the algorithm to locate the best replica of a file as well as to decide when and where replicas should be created and/or removed. This mechanism is base on an improved version of the Vivaldi algorithm that places all XtreemOS nodes into a 2D space based on communication latencies.

  • Mobile-device flavor

Regarding mobile devices, BSC is cooperating with TiD to build the embedded version of XtreemOS to be run on mobile devices. Mainly, BSC is working on the AEM and XtreemFS modules of XtreemOS-G for these devices. 

  • Applications 

Finally, BSC is porting two applications trying to take full advantage of the benefits of this new system. These applications are specweb 2005 and hmmpfam which runs on top of COMPSs a workflow manager that is also ported to its full efficiency to XtreemOS by BSC.

 

2009-09-28

XtreemFS in the news

Filed Under:

2009-08-18

Release 1.0 of XtreemFS

XtreemFS 1.0 is out. XtreemFS now supports read-only replication with failover, data center replica maps, parallel reads and writes, and a native Windows client, among other new features.

Download it at xtreemfs.org.


Watch our demo of replication and failover on youtube!

We demonstrate the XtreemFS replication and failover with an unmodified MPlayer. We play a regular movie file stored on two XtreemFS OSDs.

 

2009-08-11

XtreemOS at Linux Symposium

The Linux Symposium was held at Montreal, Canada, from July 13th to July 17th.

 

John Mehnert-Spahn (UDUS) gave a technical presentation of 45 minutes on his work "Incremental Checkpointing for Grids" on July 15th. John explained the motivation behind incremental checkpoint and its implementation in Linux SSI as well as its integration in XtreemOS. There was much interest in the implementation of incremental checkpoint. 

Linux Symposium 2009 - John

On 17th July Surbhi Chitre (INRIA) held a BOFs session for XtreemOS. She explained why XtreemOS is attractive and how it is a complete solution for grids. She also explained her work of integrating OpenVZ in XtreemOS, why it was chosen and the challenges in doing so. In her talk, she also gave a brief introduction to XtreemFS features. This was followed by a presentation by Louis Rilling of Kerlabs of Kerrighed Clusters which is supported by XtreemOS.

Linux Symposium 2009 - Surbhi

2009-07-16

XtreemOS at LinuxTag 2009

June 24-27, 2009

LinuxTag is Europe's leading event for Linux and free software. The four-day event has been taking place annually on the Berlin Exhibition Grounds since 2007. A total of 10,780 visitors from around the world attended LinuxTag 2009 in Berlin.

We presented XtreemOS and XtreemFS with a community booth at LinuxTag 2009 in Berlin.

LinuxTag-2 

During the four days of the exhibition we had a chat with about 60 visitors. For us it was very interesting to see what kind of problem the people had in mind to solve with XtreemFS. This ranged from using XtreemFS to replace VPN+Samba to video recording, editing and replication for streaming. Of course, there was quite some interest in using XtreemFS as a kind of cloud storage.


More than 10,000 visitors make LinuxTag the largest industry event in Europe - Wider-ranging conference programme and greater international participation (135 exhibiting companies and projects and around 300 speakers).

LinuxTag website

The next LinuxTag will take place from 9 to 12 June 2010 in Berlin!

2009-06-19

Get acquainted with one XtreemOS partner: Consiglio Nazionale delle Ricerche (CNR)

Consiglio Nazionale delle Ricerche (CNR)

The Italian National Research Council (CNR) is a public organization whose purpose is to carry out, promote, and improve research activities in the main sectors of knowledge, spread their results and help their transfer to the industry and their application, to further the scientific, technological, economic and social development of Italy. To this purpose, a network of CNR institutes is distributed all over Italy. To promote international cooperation and contribute to the development of key technologies, CNR participates in agreements, contracts and research projects with the European Union and with the other international organizations; among them  the CNR membership of ERCIM and of W3C. Two of the CNR institutes, ISTI and ICAR,  are involved in the development of XtreemOS.


CNR's contributions to XtreemOS

  • Scalable, Highly available services

Within XtreemOS, the CNR team is involved in the development of the highly available  services that tie up into a system the XtreemOS individual machines on a grid. CNR has developed the Service/Resource Discovery System (SRDS), that leverages on overlay networks and exploits the functionalities of the XtreemFS-F basic layer of XtreemOS,

The SRDS daemons run distributed over the nodes of the XtreemOS platform, and offer resource and service discovery services to other XtreemOS modules and applications. SRDS resolves key-based and range-based queries, over both dynamic and static information. The main  clients of the SRDS are currently the Application Execution Management (AEM) and the Job Directory Service (JDS) modules. They access information about computing resources and running jobs within a Virtual Organization.

The SRDS layered architecture employs a combination of overlay networks and query resolution algorithms to improve on previous protocols (like LDAP) in terms of scalability,  fault tolerance and flexibility. We exploit overlays based on the Distributed Hash Tree (DHT) paradigm, as well as custom networks like the RSS (also developed within XtreemOS, by VUA). The SRDS provides scalability, reliability and high availability on very large Grids, while also being able to satisfy the needs of diverse clients in terms of information management primitives and of access protocols (DIXI, RMI, Https).

CNR’s contributions

  • XtreemFS

The CNR team also participates in the development of the Object Storage Device component (OSD) of XtreemFS, the XtreemOS File System. According to the object-based storage design paradigm for parallel and distributed file systems, within XtreemFS the content of a single file is represented by one or more objects. The OSD is responsible for storing file content as a set of objects, providing access to file data interacting with the other XtreemFS components.

CNR contributes to the second development phase of XtreemFS focusing on testing it for POSIX-compliance. In this phase, the NTFS-3G test suite has been selected and is being used to assess the XtreemFS behaviour.


Istituto di Scienza e Tecnologie dell'Informazione “A. Faedo

 ISTI logoThe Institute of Information Science and Technologies (ISTI) is located in the CNR Research Area of Pisa. ISTI was constituted in September 2000, becoming fully operational in 2002, as a result of a merger between the former CNUCE and the IEI institutes of CNR. As a result, ISTI is the largest public IT research institute in Italy, enrolling about 90 permanent researchers, for a total of about 170 personell units with research fellows and PhD students.

The domain of competence of ISTI covers Information Science, related technologies and a wide range of their applications. ISTI groups into 5 main thematic areas its 16 research laboratories, covering IT application fields as different as Visual Computing from Knowledge discovery, and of course including High Performance Computing (http://www.isti.cnr.it/ResearchUnits/Labs/hpc-lab/).

Basic and applied research in the IT domain requires close integration of all components of the research community and academia with wide sectors of the industry, of business and of society at large. ISTI hosts the Italian Regional W3C Office, and is actively involved in collaborations with the academic world and in cooperative research, as well as in training and development programs, both national and international.

Istituto di Calcolo e Reti ad Alte Prestazioni del Consiglio Nazionale delle Ricerche (ICAR-CNR)

 ICAR logoThe Institute of High Performance Computing and Networking (ICAR) (http://www.icar.cnr.it) has been established on May 2001 and is operative since June 2002. The Institute is located at Rende, Cosenza (Italy) and has two branches in Naples and in Palermo. ICAR currently enrolls about 60 researchers and technicians, distributed on the three sites. ICAR performs basic and applied research in the areas of High Performance Computing and Networking and Intelligent Systems, by cooperating with the academic world and other private and public research organizations active in these areas. In addition to its research activities, the Institute works on pre-competitive development and technological transfer of research results, and carries out educational and training activities, through scholarships and research fellowships, advanced post-graduate specialization courses, and non-university higher education activities.