Matthew E.Porter
2007-03-29 16:39:53 UTC
Arkeia Users:
I am starting this thread in hopes that it will serve as an open
dialogue between Arkeia and its community. It is not meant to spread
FUD. It is focused around what I see as the upcoming future problems
- related to Arkeia and its use of file systems and inodes.
A little background about Arkeia's internal data structure is
needed to fully analyze and appreciate the situation. Arkeia server
stores a series of information in the /opt/arkeia/server/dbase
directory. This directory contains a number of sub-directories, the
most critical to this conversation is the o3dbtree directory. The
o3dbtree directory is the beginning root of a hierarchical layout for
every server/device/appliance currently being backed up by the Arkeia
server.
Examining this directory is an interesting exercise. One sees a
directory structure that lists each system being backed up. Each
backed up system has a directory structure that mimics the directory
structure actually on the system as it was backed up. In essence,
imagine a tree for every system being backed up that contains only
the directories of that system with a series of marker files (the
o3_cpnt and o3_cpnt.lck files).
This structure is intuitive and easy to understand. There is
minimal black magic. The downside is that Arkeia requires 4 inodes
per directory (remember, these are the directories on the backed up
client): 2 for the directory itself, 1 for o3cpnt file, and 1 for the
o3cpnt.lck lock file. The downside is that you need a file system
that can handle this. The official stance from Arkeia is "In most
cases, if the disk space is enough to hold the database, then the i-
node number is enough. But, if you have a server which contains lot
of directories with few files you may have to change the i-node
configuration for your index database." (http://www.arkeia.com/
support/faq/networkbackup.php)
I disagree and see a significant issue approaching with Arkeia.
The amount of inodes has a limit on most major file systems. As
Linux is the recommended platform for deployment of Arkeia (server),
the issue is even more direct with the recent happenings in the file
system space. For years, Arkeia has recommended Reiserfs as the
solution to this problem. Unfortunately, the future Reiserfs is in a
relatively unknown and looming state. Consider the following:
- SuSE Linux has dropped ReiserFS as the default file system. See
http://linux.wordpress.com/2006/09/27/suse-102-ditching-reiserfs-as-
it-default-fs/ for a lengthy discussion
- Reiser 3 is in maintenance mode
- Reiser 4 does not appear that it will gain acceptance into the kernel
- The future of Hans Reiser and his company are in doubt given the
recent problems. See http://kcbs.com/pages/319259.php?
contentType=4&contentId=386120 if you have no clue why.
These are issues that will begin impacting more and more Arkeia
users as drives expand in size. Arkeia's competitors are noting that
this will impact the near term usage of Arkeia's users. With
consumer PCs shipping with drives as large as 1 TB (http://
tinyurl.com/2zdaj6), how can this not impact Arkeia deployments?
The question pops up "Doesn't Arkeia 6 solve these problems with
the media server deployment capabilities?". I do not fully know the
answer but believe that it does not. The media servers are
responsible for storage of the backed up data; however, the central
backup server still is responsible for the index directory (the
location of the o3dbtree structure). If my understanding is correct,
the ability for Arkeia to backup more data has been increased, and
this could actually cause more users to hit this limit.
(This skips over the current problems with this structure. For
example, it takes approximately 40 hours for an rsync to run on our /
opt/arkeia/server directory which has 42 GB of data in it. The
server is utilizing a RAID10 configuration with 15k Serial Attached
SCSI drives. It is nearly *impossible* for to backup this directory
through traditional methods.)
This was not mean to portray the situation as doom and gloom.
This does *NOT* mean people should run out and abandon Arkeia. In
reality, I am hoping for quite the opposite. I am hoping someone out
there has an answer to this. At bare minimum, I am hoping this will
open a dialogue with Arkeia. As a customer paying annual support, I
am hoping to hear that Arkeia is looking to the future and these issues.
Arkeia is a great piece of software, and our company has a
significant investment in it. We have developed custom applications
to provide features that we need, including integration with our
management system. We are happy Arkeia users. I want this statement
to still be true in 2 to 5 years.
Cheers,
Matthew
---
Matthew E. Porter
Contegix
Beyond Managed Hosting(r) for Your Enterprise
I am starting this thread in hopes that it will serve as an open
dialogue between Arkeia and its community. It is not meant to spread
FUD. It is focused around what I see as the upcoming future problems
- related to Arkeia and its use of file systems and inodes.
A little background about Arkeia's internal data structure is
needed to fully analyze and appreciate the situation. Arkeia server
stores a series of information in the /opt/arkeia/server/dbase
directory. This directory contains a number of sub-directories, the
most critical to this conversation is the o3dbtree directory. The
o3dbtree directory is the beginning root of a hierarchical layout for
every server/device/appliance currently being backed up by the Arkeia
server.
Examining this directory is an interesting exercise. One sees a
directory structure that lists each system being backed up. Each
backed up system has a directory structure that mimics the directory
structure actually on the system as it was backed up. In essence,
imagine a tree for every system being backed up that contains only
the directories of that system with a series of marker files (the
o3_cpnt and o3_cpnt.lck files).
This structure is intuitive and easy to understand. There is
minimal black magic. The downside is that Arkeia requires 4 inodes
per directory (remember, these are the directories on the backed up
client): 2 for the directory itself, 1 for o3cpnt file, and 1 for the
o3cpnt.lck lock file. The downside is that you need a file system
that can handle this. The official stance from Arkeia is "In most
cases, if the disk space is enough to hold the database, then the i-
node number is enough. But, if you have a server which contains lot
of directories with few files you may have to change the i-node
configuration for your index database." (http://www.arkeia.com/
support/faq/networkbackup.php)
I disagree and see a significant issue approaching with Arkeia.
The amount of inodes has a limit on most major file systems. As
Linux is the recommended platform for deployment of Arkeia (server),
the issue is even more direct with the recent happenings in the file
system space. For years, Arkeia has recommended Reiserfs as the
solution to this problem. Unfortunately, the future Reiserfs is in a
relatively unknown and looming state. Consider the following:
- SuSE Linux has dropped ReiserFS as the default file system. See
http://linux.wordpress.com/2006/09/27/suse-102-ditching-reiserfs-as-
it-default-fs/ for a lengthy discussion
- Reiser 3 is in maintenance mode
- Reiser 4 does not appear that it will gain acceptance into the kernel
- The future of Hans Reiser and his company are in doubt given the
recent problems. See http://kcbs.com/pages/319259.php?
contentType=4&contentId=386120 if you have no clue why.
These are issues that will begin impacting more and more Arkeia
users as drives expand in size. Arkeia's competitors are noting that
this will impact the near term usage of Arkeia's users. With
consumer PCs shipping with drives as large as 1 TB (http://
tinyurl.com/2zdaj6), how can this not impact Arkeia deployments?
The question pops up "Doesn't Arkeia 6 solve these problems with
the media server deployment capabilities?". I do not fully know the
answer but believe that it does not. The media servers are
responsible for storage of the backed up data; however, the central
backup server still is responsible for the index directory (the
location of the o3dbtree structure). If my understanding is correct,
the ability for Arkeia to backup more data has been increased, and
this could actually cause more users to hit this limit.
(This skips over the current problems with this structure. For
example, it takes approximately 40 hours for an rsync to run on our /
opt/arkeia/server directory which has 42 GB of data in it. The
server is utilizing a RAID10 configuration with 15k Serial Attached
SCSI drives. It is nearly *impossible* for to backup this directory
through traditional methods.)
This was not mean to portray the situation as doom and gloom.
This does *NOT* mean people should run out and abandon Arkeia. In
reality, I am hoping for quite the opposite. I am hoping someone out
there has an answer to this. At bare minimum, I am hoping this will
open a dialogue with Arkeia. As a customer paying annual support, I
am hoping to hear that Arkeia is looking to the future and these issues.
Arkeia is a great piece of software, and our company has a
significant investment in it. We have developed custom applications
to provide features that we need, including integration with our
management system. We are happy Arkeia users. I want this statement
to still be true in 2 to 5 years.
Cheers,
Matthew
---
Matthew E. Porter
Contegix
Beyond Managed Hosting(r) for Your Enterprise