MooseFS > 12. MooseFS. Architecture.
Author: Aleksandr Chirkov
Aug. 15, 2014, 1:01 a.m.

Recommended way to organize MooseFS cluster could be next:

MFSMaster  -- > two metaloggers  -- > few chunk servers  -- > client machines. MooseFS developers suggest use HA solution based on ucarp. From myself I could add that ucarp is not the best solution for production, but anyway in most cases it works.

For aceleration data servers (chunk servers) use RAID 0 massives. RAID 1/5 massives are redundant due to MooseFS fault-tolerance. With disks amount under chunk servers increase IO operations speed.

ARCHITECTURE

 

MooseFS consists of four components:

  • Managing server (master server) – a single machine managing the whole filesystem, storing metadata for every file (information on size, attributes and file location(s), including all information about non-regular files, i.e. directories, sockets, pipes and devices).
  • Data servers (chunk servers) - any number of commodity servers storing files data and synchronizing it among themselves (if a certain file is supposed to exist in more than one copy).
  • Metadata backup server(s) (metalogger server) - any number of servers, all of which store metadata changelogs and periodically downloading main metadata file; so as to promote these servers to the the role of the Managing server when primary master stops working.
  • Client computers that access (mount) the files in MooseFS - any number of machines usingmfsmount process to communicate with the managing server (to receive and modify file metadata) and with chunkservers (to exchange actual file data).

mfsmount is based on the FUSE mechanism (Filesystem in USErspace), so MooseFS is available on every Operating System with a working FUSE implementation (Linux, FreeBSD, MacOS X, etc.)

 

Metadata is stored in the memory of the managing server and simultaneously saved to disk (as a periodically updated binary file and immediately updated incremental logs). The main binary file as well as the logs are synchronized to the metaloggers (if present).
File data is divided into fragments (chunks) with a maximum of 64MiB each. Each chunk is itself a file on selected disks on data servers (chunkservers). 
High reliability is achieved by configuring as many different data servers as appropriate to realize the "goal" value (number of copies to keep) set for the given file.

 

HOW THE SYSTEM WORKS

 

All file operations on a client computer that has mounted MooseFS are exactly the same as they would be with other file systems. The operating system kernel transfers all file operations to the FUSE module, which communicates with the mfsmount process. The mfsmount process communicates through the network subsequently with the managing server and data servers (chunk servers). This entire process is fully transparent to the user.


mfsmount communicates with the managing server every time an operation on file metadata is required:

  • creating files
  • deleting files
  • reading directories
  • reading and changing attributes
  • changing file sizes
  • at the start of reading or writing data
  • on any access to special files on MFSMETA

mfsmount uses a direct connection to the data server (chunk server) that stores the relevant chunk of a file. When writing a file, after finishing the write process the managing server receives information frommfsmount to update a file's length and the last modification time.

Furthermore, data servers (chunk servers) communicate with each other to replicate data in order to achieve the appropriate number of copies of a file on different machines.

linux
moosefs
distributed-file-systems
architecture
Clomid Pour Carte Des Hommes Propecia Online Without A Prescription [url=http://costofcial.com]online pharmacy[/url] Buy Alli Starter Pack Levitra 10 Mg Vidal Comprar Viagra En Linea Achetez Cialis isotretinoin shop online pharmacy Where To Order Provera Cialis Laboratoire Lilly Viagra Effetti Forum Cialis 10 Mg Pharmacie Comprar Cialis Barcelona Online Mifepristone And Misoprostol For Sale http://costofcial.com - cialis price Real Best Viagra Online Pharmacy Drugstore
Propecia Und Viagra My Pharmacy 365 Anything Otc Like Viagra Or Cialis [url=http://costofcial.com]viagra cialis[/url] Levitra Scheda Tecnica Comparaison Cialis Et Levitra Viagra Generica En Espana Zithromax Generic Cost cialis Provera Medicine Deutschlnd Drug Levitra Acquistare Impotenza Il Cialis Scade http://costofcial.com - cialis Prix Cialis En Pharmacie
buy cialis in egypt [url=http://cialisle.com]generic cialis[/url] why would a woman take cialis http://cialisle.com tomar cialis 20 mg
sildenafil and viagra difference [url=http://viagrabs.com/]generic viagra online[/url] how to avoid a viagra headache buy viagra online other indications for viagra
cheap ed meds online ed medicines generic for erectile dysfunction ed medications best ed medication what are the best generic ed drugs online erectile dysfunction medications ed medications compared cheap ed medication ed medications online ed drugs online best erectile dysfunction medication in canada ed drugs meds for ed erectile dysfunction medications sold in canada erectile dysfunction drugs generic ed drugs cheap erectile dysfunction drugs-canada canadian erectile dysfunction drugs ED medication order erectile dysfunction medication generic drugs for erectile dysfunction erectile meds ed drug cost comparison buy erectile dysfunction drugs generic ed medication erectile dysfunction medication prices ed meds on line cheap ed drugs best erectile dysfunction drugs for sale online erectile dysfunction drugs online meds for erectile dysfunction generic ed meds erectile dysfunction drugs canada ed drug best ed drug best deal on ED meds cheap ed meds medication from canada ed erectile dysfunction generic name for ED drugs buy erectile dysfunction drugs online cheap erectile dysfunction medication generic erectile dysfunction medications erectile dysfunction drugs comparison impotence drugs online common drugs for ed generic impotence drugs ed meds cost effective ed drugs canadian erectile dysfunction drugs rosa impex pvt ltd levitra vs viagra vs cialis amoxicillin no prescription needed but generic lexapro online cheap pfizer viagra 100mg price sildenafil citrate 100mg paypal online pharmacy no prescription viagra samples overnight generic cialis without prescription
what is the dose for cialis [url=http://cialisle.com]cialis[/url] efficacy of generic cialis buy cialis online long term usage cialis
how much viagra is safe to take in one day [url=http://www.viagrabs.com/]viagra online[/url] ray peat viagra http://viagrabs.com/ salt of viagra

Leave a comment:

Name
Python
Web development
Review Linux
Administration
Provisioning CVS
Windows AWS
Continuous Integration
NoSql Logging Web Servers FileSystems Scala
accurev ansible apach2 apache api application approaches architecture archivation argparse arguments artifactory automation awk aws backup bash batch-scripting beaver block build built-in caching call captcha cartridge case certificates cgi chain client cloud cloudformation cluster cmd coding collections command-line commands compression conditionals conversion convert cookbook copy counter cron crud css cut cygwin daemon datatype date dd debpackage decorator delattr deploy deque dict distributed-file-systems django DNS domain driver dropbox dump elasticcache elasticsearch encryption exit ext4 extra-tags failover file filename filters for form format freetds functions gerrit getattribute gid git globals glusterfs hardware hook iam indexing inheritance init install job-interview jquery ldap linux list locals logging logs logstash lookup magicmethods mail main-menu metadata metalogger mongodb moosefs mount mssql multiplatform multithreading mysql netcat nginx nosql open-ssh openldap openshift os packaging parse partition path pattern patterns permissions pid pil pip pipe playbook pool post post-commit processes production provisioning proxy putty python python-mysql recursively redirection redis register replication repr restore return review rotation scala script search selenium server setattr settings setup shipper signals singleton slots snapshot socket splunk ssh ssh-key ssl storage str string style subprocess sugar super switch syntactic syntaxhighlighter systeminfo tail tar templatetags time tls tune2fs tuple ubuntu unicode unique unix unixodbc usage usecases uuid uwsgi variable vars version vi virtualenv volume web web-server windows with_items __getattr__