Reviews about GNU/Linux software and hardware

• Section menu
  • Top
  • The Xen hypervisor
  • The SVK souce control system
  • Software IP phones
  • aptitude
  • Debian distribution
  • 3ware simulates a SCSI disk subsystem
  • Conexant USB drivers

aptitude (040709)

It is so much better than other ways of dealing with APT based dependencies. Only Synaptic is comparable, and it has a GUI, and is less finely tunable to do things manually.

Things that suck:

• The docs are poor (but I have noticed someone has contributed some recently, but I haven't had a look yet) and poorly formatted.
• Like all DPKG/APT based things, it wastes a lot of memory and CPU time by loading the whole package list in memory, and reloading it almost on every operation, which can take several minutes even on a fairly substantial PC.
• Doing simple operations like tagging a package for installation can take several seconds of CPU time even on a fairly substantial CPU.
• There is a lot of bizarre confusion between the full screen sessions/panels and the occasional popup windows. That there are full screen sessions/pannels switchable using F6/F7 is not well documented or obvious.
• When limiting, the filtering happens on both installed and non installed versions; the filtering is only possible on installed package names.
• Cannot display the list of files in a package.
• Cannot sort by install date or by created date.
• Cannot sort by download sized.
• Cannot limit by date or size.
• Package information does not display some attributes on which limiting is possible, like edition (~Aedition).
• The multiple window aspect is poorly documented, and is ungainly without a window manager.

Debian distribution (040807)

My overall opinion is that it is tolerably good technically, even if with some important problems listed later. The best thing is that is “politically correct”.

RedHat, SUSE or Mandrake (and even Slackware) could go the way of SCO; a change of ownership or management or business plan might make them switch to offering a proprietary distribution containing mostly free software, like other companies do. This cannot imaginably happen with Debian.

Debian seems very much targeted at hard core UNIX people with access to lots of bandwidth, running either bleeding edge desktops for themselves or very stable services on old servers.

There are some distributions derived from Debian, like Knoppix, Libranet, Mepis, User Linux, Progeny, Munjoy, Bonzai, or even Linspire or Xandros, that are much better suited for non-advanced desktop users.

The Debian project does many things, but the main activity seems to be maintaining a vast collection of packages. It used to be that it was about releasing a distribution, but for the past few years this has not happened very much.

Another social aspect of the Debian project is that it seems that many of its members are somewhat obsessive and that perhaps as a result of this the release of an official distribution has been made conditional upon a large number of obsessively defined conditions, like being completely available on a large number of platforms and a long period of QA.

Since each of the strictures taken by itself is a good idea, and has enough vocal defenders, in practice none of them can be dropped, but their overall effect is near paralysis as far as official releases are concerned.

Other social strictures (more than structures) of Debian are fairly interesting too. Debian is a cathedral style project in which the following rules apply:

• Only project members can contribute or maintain new packages (in 2004 there were more than 2,000 members maintaining more than 10,000 packages).
• Members are inducted into the project by cooptation after some period of apprenticeship in the often complicated procedures for dealing with packaging in Debian.
• Once a member has volunteered to maintain a package, he remains the maintainer until he gives up, no matter whether this is done well or badly, not that anybody checks that. There are conceivable ways of forcibly changing maintainership of a package from a member to another, but they are extremely exceptional.
• If a member does not like how another member is maintaining a package, in practice it is extremely difficult to submit a differently packaged version of the same sources.

Also, I think that as most Debian developers already have Debian installed and are online via high speed lines, most are just happy to contribute to and keep updating via the package collection, and the bother of actually releasing a distribution does not excite them much.

Other projects are releasing distributions based on the Debian package collection, and their are more focused on releasing distributions more frequently and targeted to a greater range of potential uses.

Probably in effect if not in theory the Debian project will become like the Mozilla project, whose purpose is to develop a code base, and release test snapshosts of that code base, not to release a finished application.

Things that suck:

• When installing or updating a package that contains a daemon, Debian will start the daemon automatically and it is very hard or impossible to prevent this; not only this is very annoying (for example, on the first installation the daemon will often start with the default configuration) but it can lead to serious security problems, like under MS Windows, which also starts too many services by default.
• There is a maze of little scripts that edit and update configurations files, and they are usually poorly written and often break easily, and are often hard to override.
• Many packages have pre/port install/remove autorun scripts that do lots of things (and are often not very well written). At one given moment my PC had 3,092 packages installed, with 4,287 such scripts, totaling 147,890 lines of code (usually shell of Perl source). 200 scripts were more than 100 lines long, and 105 were more than 500 lines long.
• A lot of X11 programs are dumbly installed outside the /usr/X11R6/ subhierarchy. This is required by the official Debian packaging policy.
• The lack of frequent official releases or even unofficial ISO images means that online updating is usually the best way to stay current, and the size and frequency of the metadata and package updates make it hard for non broadband users to stay current. While Debian can be maintained over a dialup line, or even without an online connection, it is rather unpleasant except for very stable servers.
• The base package tool, and some of the package management system, and some of the package policies really, really suck:
  • The DPKG database is very, very inefficient.
  • The package management tools read the whole database into memory every time they are invoked, and this is very slow.
  • When updating the lists of available packages (Packages.gz), the lists are often so large even compressed that it can take a long time over a dialup line, and one has to download the whole list even if only a few package descriptions in it have been changed.
  • When installing a lot of packages the dependency management tools first download all, and then install them, instead of only doing incremental downloads of those needed at one particular moment.
  • The package format is based on an ar archive containing tar archives, which makes it hard to install a package without first spooling it to a temporary directory.
  • APT cannot handle the case where one wants to install packages from multiple architectures, or install multiple versions of the same package, which means that the version name is often appended to the package name.
  • It is somewhat unnatural to have two packages with the same name but different versions installed, even when they do not overlap. The result is that for many packages where one wants multiple versions to be installed at the same time the version number has been duplicated in the package name, which is so stupid.
  • One cannot have packages with the same name but for different architectures installed at the same time, unless one makes the architecture name part of the package name, just like for versions.
  • There are very many package management commands.
  • It is somewhat complicated to create a source package, so much so that there is a long apprenticeship system to be officially recognized as a package creator.
  • The source package is not based on a totally pristine source archive; the source archive must usually be renamed.
  • The package archive structure is fairly complicated and it's hard to do selective downloading except via the dependency management frontends, which are somewhat complex.
  • By Debian policy, all library packages have a name beginning with lib, even if they are really subpackages of a package with a name that does not begin with lib. This means that for example openssl and libssl are really part of the same group of packages, being compiled from the same source package, even if they have seem to have rather different names.
  Note that they also miss some useful functionality, but that is something that is being fixed.
• The whole package management and APT story seems best suited to those that have very large computers attached to very fast network links that need to be stable and only need to be updated rarely; for example, servers at large academic and commercial institutions. There seems to be a bit of implicit bias against, or perhaps just benign neglect, home users on modem links.

3ware simulates a SCSI disk subsystem

SCSI drives and host adapters usually support mailboxing with tagged queueing, and these give a very large performance advantage when several programs read a disk concurrently.

Unfortunately SCSI disk drives tend to cost several times more than ATA disk drives os equivalent capacity and otherwise similar performance.

3ware and other companies have spotted this as an opportunity and have designed host adapters that allow ATA drives to be used for RAIDs.

3ware in particular have one of these ATA RAID host adapters that simulates a SCSI host adapter. The important feature is that it has an onboard processor that handles queues of requests and the Linux driver supports that. In other words it does mailboxing with tagged queueing with ATA drives.

I have tried a 3ware 7410 model with two IBM 120GB drives; the drives were arranged as a RAID 0 (striped) volume, under Linux kernel 2.4.18. Each disk is capable of a sequential transfer rate of about 40MB/s with a single process reading from it, which falls to 8MB/s and lower when multiple processes read from it concurrently using a standard ATA host adapter.

When reading with a single process using the 3ware 7410 the aggregate transfer rate was about 80MB/s (because of striping across the two disks).

When reading with multiple process using the 3ware 7410, the aggregate transfer rate was near to 40MB/s, but it stayed near to 40MBs even if 10 or 20 processes were reading concurrently. This indicates that the 3ware does perform mailboxing with tagged queueing.

This is the same performance that is obtained by using a SCSI host adapter. But while the 3ware 7410 and an equivalent SCSI host adapter cost much the same (around $230), the ATA disk costs less than one third ($190) of a similar (smaller but faster) SCSI disk (about $680). In particular, the price of the 3ware card is half the difference in price between the ATA and SCSI disk drive.

The only downside to the 3ware cards is that they really require a PCI64 slot, even if they are comaptible with PCI32 ones, because of the very high transfer rates they support. Unfortunately PCI64 slots are present almost only on fairly expensive SMP server class motherboards. But then striped RAIDs can easily exceed the bandwidth of the PCI32 bus.

Conexant USB drivers

I have had the misfortune of buying a Zoom 5510A USB ADSL modem, and to make it work under Linux.

I have felt unfortunate because this has meant that I have had to have a look at the sources for the Linux drivers for this chipset, and to me they range from revolting to the merely appalling.

It is hard to convey just how outraged I am that what seems to me rubbish like the cxacru sources exist and infest my system. I have with considerable effort forced myself to look at them in detail, and even started rewriting them.

Among the numerous aspects that I think are ignorant or silly (from the makefile to the scripts, from the sources of the user space utility to the layout, and so on), one seems to stand out: the nontrivial API between the driver and its controlling utility is defined twice, with different symbols, in the driver and in its controlling utility, and the two ways just happen to be equivalent.

The saddest aspect of this all seems to me that the authors of the sources probably are individuals with lots of potential to be good programmers, because after all the drivers mostly work.

I think that writing something fairly complex well requires skills, writing something fairly complex really badly and still make it work requires extraordinary capabilities.

Finally the speedtch project has created a completely new (and much better written) cxacru driver which is from version 2.6.13 a standard part of the mainline kernel.