Comments on: Comparing NTFS-3G to ZFS-FUSE for FUSE Performance http://www.csamuel.org/2007/04/25/comparing-ntfs-3g-to-zfs-fuse-for-fuse-performance Computers, science, archaeology and other random burblings Sun, 22 Aug 2010 15:02:30 +0000 hourly 1 http://wordpress.org/?v=3.0.1 By: Tried ZFS on Linux? – LINUX For You Magazine http://www.csamuel.org/2007/04/25/comparing-ntfs-3g-to-zfs-fuse-for-fuse-performance/comment-page-1#comment-56698 Tried ZFS on Linux? – LINUX For You Magazine Thu, 09 Jul 2009 10:58:26 +0000 http://www.csamuel.org/2007/04/25/comparing-ntfs-3g-to-zfs-fuse-for-fuse-performance/#comment-56698 [...] ZFS on FUSE is a project under development by Ricardo Manuel da Silva Correia, a computer engineering student, and is sponsored by Google as part of Google Summer of Code 2006. So after completion of this project, ZFS will have a port on the FUSE framework, which effectively will mean operating systems like Linux can use ZFS. A rough performance comparison of ZFS on FUSE with NTFS-3G, XFS and JFS can be found at www.csamuel.org/2007/04/25/comparing-ntfs-3g-to-zfs-fuse-for-fuse-performance. [...] [...] ZFS on FUSE is a project under development by Ricardo Manuel da Silva Correia, a computer engineering student, and is sponsored by Google as part of Google Summer of Code 2006. So after completion of this project, ZFS will have a port on the FUSE framework, which effectively will mean operating systems like Linux can use ZFS. A rough performance comparison of ZFS on FUSE with NTFS-3G, XFS and JFS can be found at http://www.csamuel.org/2007/04/25/comparing-ntfs-3g-to-zfs-fuse-for-fuse-performance. [...]

]]> By: How to implement ZFS on FUSE | techinterplay http://www.csamuel.org/2007/04/25/comparing-ntfs-3g-to-zfs-fuse-for-fuse-performance/comment-page-1#comment-56404 How to implement ZFS on FUSE | techinterplay Thu, 18 Jun 2009 13:01:48 +0000 http://www.csamuel.org/2007/04/25/comparing-ntfs-3g-to-zfs-fuse-for-fuse-performance/#comment-56404 [...] use ZFS. A rough performance comparison of ZFS on FUSE with NTFS-3G, XFS and JFS can be found at http://www.csamuel.org/2007/04/25/comparing-ntfs-3g-to-zfs-fuse-for-fuse-performance [...] [...] use ZFS. A rough performance comparison of ZFS on FUSE with NTFS-3G, XFS and JFS can be found at http://www.csamuel.org/2007/04/25/comparing-ntfs-3g-to-zfs-fuse-for-fuse-performance [...]

]]> By: Between the Lines mobile edition http://www.csamuel.org/2007/04/25/comparing-ntfs-3g-to-zfs-fuse-for-fuse-performance/comment-page-1#comment-26078 Between the Lines mobile edition Tue, 29 Apr 2008 14:58:17 +0000 http://www.csamuel.org/2007/04/25/comparing-ntfs-3g-to-zfs-fuse-for-fuse-performance/#comment-26078 [...] However, initial performance tests conducted in 2007 indicate that with continued development, NTFS may actually have some potential as a mainstream Linux filesystem. However, in order to get any sort of community buy-in, the full Microsoft specifications for the filesystem primitives and metadata would have to be released - and perhaps even a full GPL2 release of Microsoft’s actual NTFSv5 drivers on Windows Server 2008. While this sounds like something of a pipe dream, Microsoft has already stated its intention to cooperate with the Opern Source community by stating its Interoperability Principles in February of 2008, and has already published 45,000 pages of documentation detailing the functions and workings of its network protocols, so such a gesture is not completely out of the question. [...] [...] However, initial performance tests conducted in 2007 indicate that with continued development, NTFS may actually have some potential as a mainstream Linux filesystem. However, in order to get any sort of community buy-in, the full Microsoft specifications for the filesystem primitives and metadata would have to be released – and perhaps even a full GPL2 release of Microsoft’s actual NTFSv5 drivers on Windows Server 2008. While this sounds like something of a pipe dream, Microsoft has already stated its intention to cooperate with the Opern Source community by stating its Interoperability Principles in February of 2008, and has already published 45,000 pages of documentation detailing the functions and workings of its network protocols, so such a gesture is not completely out of the question. [...]

]]> By: Chris Samuel http://www.csamuel.org/2007/04/25/comparing-ntfs-3g-to-zfs-fuse-for-fuse-performance/comment-page-1#comment-7896 Chris Samuel Thu, 05 Jul 2007 08:39:53 +0000 http://www.csamuel.org/2007/04/25/comparing-ntfs-3g-to-zfs-fuse-for-fuse-performance/#comment-7896 Er, booting from ZFS/FUSE under Linux <a href="http://groups.google.com/group/zfs-fuse/browse_thread/thread/3e781ace9de600bc/230ca0608235e216">already works</a> folks! Riccardo commented on that feat saying: <blockquote><em>As a side note, it's interesting to know that Linux is the first operating system that can boot from RAID-1 0, RAID-Z or RAID-Z2 ZFS pools (Solaris can only boot from single-disk or RAID-1 pools)</em></blockquote> :-) Er, booting from ZFS/FUSE under Linux already works folks!

Riccardo commented on that feat saying:

As a side note, it’s interesting to know that Linux is the first operating system that can boot from RAID-1 0, RAID-Z or RAID-Z2 ZFS pools (Solaris can only boot from single-disk or RAID-1 pools)

:-)

]]> By: Szabolcs Szakacsits http://www.csamuel.org/2007/04/25/comparing-ntfs-3g-to-zfs-fuse-for-fuse-performance/comment-page-1#comment-7895 Szabolcs Szakacsits Thu, 05 Jul 2007 08:30:31 +0000 http://www.csamuel.org/2007/04/25/comparing-ntfs-3g-to-zfs-fuse-for-fuse-performance/#comment-7895 No need for in-kernel NTFS support to boot Linux from NTFS. Grub supports this natively (ZFS too) and NTFS-3G implemented bmap which is needed by LILO. Several distributions use NTFS-3G for root file system, for example WUBI (Windows Ubuntu Installer). When the kernel booted with an initrd or ramfs then it can mount a file system and pivotroot to it to be the root file system. There are some minor issues e.g. during shutdown in which order the subsystems and processes are terminated but they are solvable and being worked on. No need for in-kernel NTFS support to boot Linux from NTFS. Grub supports this natively (ZFS too) and NTFS-3G implemented bmap which is needed by LILO. Several distributions use NTFS-3G for root file system, for example WUBI (Windows Ubuntu Installer). When the kernel booted with an initrd or ramfs then it can mount a file system and pivotroot to it to be the root file system.

There are some minor issues e.g. during shutdown in which order the subsystems and processes are terminated but they are solvable and being worked on.

]]> By: Mitch 74 http://www.csamuel.org/2007/04/25/comparing-ntfs-3g-to-zfs-fuse-for-fuse-performance/comment-page-1#comment-7894 Mitch 74 Thu, 05 Jul 2007 07:37:10 +0000 http://www.csamuel.org/2007/04/25/comparing-ntfs-3g-to-zfs-fuse-for-fuse-performance/#comment-7894 you can boot from an NTFS partition using ntfs-3g, because the kernel can read NTFS by default (thus load its image, then mount the boot partition read only, then load ntfs-3g, then remount it as read/write). If there is a read-only implementation of ZFS in-kernel, then it can already boot. you can boot from an NTFS partition using ntfs-3g, because the kernel can read NTFS by default (thus load its image, then mount the boot partition read only, then load ntfs-3g, then remount it as read/write). If there is a read-only implementation of ZFS in-kernel, then it can already boot.

]]> By: Chris Norton http://www.csamuel.org/2007/04/25/comparing-ntfs-3g-to-zfs-fuse-for-fuse-performance/comment-page-1#comment-7879 Chris Norton Wed, 04 Jul 2007 16:01:02 +0000 http://www.csamuel.org/2007/04/25/comparing-ntfs-3g-to-zfs-fuse-for-fuse-performance/#comment-7879 That's very interesting. Sounds like we need to see about properly optimising the FUSE port then! :) Of course, having a proper in-kernel implementation would still be the best solution as then you'd be able to boot of of ZFS and therefore use it exclusively. Still, it'd be nice to be able to store at least my data files on a ZFS partition. That’s very interesting. Sounds like we need to see about properly optimising the FUSE port then! :)

Of course, having a proper in-kernel implementation would still be the best solution as then you’d be able to boot of of ZFS and therefore use it exclusively. Still, it’d be nice to be able to store at least my data files on a ZFS partition.

]]> By: Szabolcs Szakacsits http://www.csamuel.org/2007/04/25/comparing-ntfs-3g-to-zfs-fuse-for-fuse-performance/comment-page-1#comment-7802 Szabolcs Szakacsits Tue, 26 Jun 2007 13:34:23 +0000 http://www.csamuel.org/2007/04/25/comparing-ntfs-3g-to-zfs-fuse-for-fuse-performance/#comment-7802 Hi Chris, I don't think the amount of code which needs to be rewritten under a different licence for optimization to be included in the kernel would be significant compared to a full rewrite which is practically impossible and would take about "forever". Let's say full rewrite vs optimization: 100,000 vs 1000 lines of code. In fact, some of the code which is needed to be in the kernel are already under GPL2 (e.g. checksum, compression). Hi Chris,

I don’t think the amount of code which needs to be rewritten under a different licence for optimization to be included in the kernel would be significant compared to a full rewrite which is practically impossible and would take about “forever”. Let’s say full rewrite vs optimization: 100,000 vs 1000 lines of code. In fact, some of the code which is needed to be in the kernel are already under GPL2 (e.g. checksum, compression).

]]> By: Chris Samuel http://www.csamuel.org/2007/04/25/comparing-ntfs-3g-to-zfs-fuse-for-fuse-performance/comment-page-1#comment-7801 Chris Samuel Tue, 26 Jun 2007 11:27:53 +0000 http://www.csamuel.org/2007/04/25/comparing-ntfs-3g-to-zfs-fuse-for-fuse-performance/#comment-7801 Hi Szabolcs, thanks for that information! Unfortunately because of ZFS being under CDDL it's unlikely that any of its code will make it into the kernel for the moment (unless it's derived from the <a href="http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/grub/grub-0.95/stage2/fsys_zfs.c">GPL licensed bits in GRUB</a>). Hi Szabolcs, thanks for that information!

Unfortunately because of ZFS being under CDDL it’s unlikely that any of its code will make it into the kernel for the moment (unless it’s derived from the GPL licensed bits in GRUB).

]]> By: Szabolcs Szakacsits http://www.csamuel.org/2007/04/25/comparing-ntfs-3g-to-zfs-fuse-for-fuse-performance/comment-page-1#comment-7785 Szabolcs Szakacsits Mon, 25 Jun 2007 10:21:53 +0000 http://www.csamuel.org/2007/04/25/comparing-ntfs-3g-to-zfs-fuse-for-fuse-performance/#comment-7785 Nice to see these benchmarks! :-) NTFS-3G is indeed completely unoptimized. I shortly mentioned the current major bottlenecks on http://lwn.net/Articles/238812/ but of course there are many more. The similarities between the performance of XFS and NTFS-3G could be that I use XFS and currently that's the "baseline" performance during NTFS-3G development. Anything much worse than that is considered to be a usability bug, not a performance problem, so it gets more attention. The comparison is valid when I/O is the real bottleneck, not the CPU. If the processor is too slow compared to the disk (e.g. embedded devices or high I/O throughput servers) then user space file systems will suffer a lot since the performance support infrastucture in the kernel and FUSE isn't developed, optimized yet. The new, unfinished, unoptimized ntfs-3g block allocator since version 1.328 helps if the volume is fragmented. I also noticed that Microsoft's NTFS block allocator is fairly inefficient. About the "scalability" (40 -> 120 GB disk experience). I think the reason for the high CPU usage is what I mentioned above: the bigger the disk the faster, so the CPU can be used more which results higher CPU usage. Thanks Mitch74 for the ideas. Automatic detection of FS code page: highly OS and environment specific. If the distribution, OS vendor sets it up properly before mounting an NTFS volume then the driver will work fine without the usage of 'locale=' workaround mount option. As for zfs-fuse, since data should stay in the kernel hence some of zfs code should also go there (e.g. end-to-end checksumming). Nice to see these benchmarks! :-)

NTFS-3G is indeed completely unoptimized. I shortly mentioned the current major
bottlenecks on http://lwn.net/Articles/238812/ but of course there are many more.

The similarities between the performance of XFS and NTFS-3G could be that I use
XFS and currently that’s the “baseline” performance during NTFS-3G development.
Anything much worse than that is considered to be a usability bug, not a performance
problem, so it gets more attention.

The comparison is valid when I/O is the real bottleneck, not the CPU. If the
processor is too slow compared to the disk (e.g. embedded devices or high
I/O throughput servers) then user space file systems will suffer a lot
since the performance support infrastucture in the kernel and FUSE isn’t
developed, optimized yet.

The new, unfinished, unoptimized ntfs-3g block allocator since version 1.328
helps if the volume is fragmented. I also noticed that Microsoft’s NTFS block
allocator is fairly inefficient.

About the “scalability” (40 -> 120 GB disk experience). I think the reason for the
high CPU usage is what I mentioned above: the bigger the disk the faster, so the
CPU can be used more which results higher CPU usage. Thanks Mitch74 for the
ideas.

Automatic detection of FS code page: highly OS and environment specific. If the
distribution, OS vendor sets it up properly before mounting an NTFS volume then
the driver will work fine without the usage of ‘locale=’ workaround mount option.

As for zfs-fuse, since data should stay in the kernel hence some of zfs code
should also go there (e.g. end-to-end checksumming).

]]>