Portability Blog (Entries tagged as unix)

Perl exec

nospam@example.com (Daniel Fischer) — Wed, 23 Jun 2010 08:34:21 +0200

Perl is a popular scripting language in the UNIX world, and there are several ports available for Windows. If your UNIX project uses perl, then porting that code to Windows is fairly painless. Perl doesn't attempt to be entirely portable, but most of the differences are quite obvious.

One that is apparently not obvious is perl's exec function. I've seen this happen to two different projects recently, in short succession. The symptom was that a process was started by a perl script, but no output from it was recorded on Windows. On UNIX, everything worked as expected.

This turns out to be not very complicated. On UNIX, exec is implemented in terms of the execl()/execv() family of syscalls. This means the child process replaces the parent, and any process waiting for that parent automatically waits for the child instead. On Windows, there is no such thing. Windows perl implements exec by spawning a child process and returning control to the parent immediately, which then exits. Any process waiting for the parent regains control instantly and is left unaware of the child. A script written for UNIX will assume that the work is done before it has even started.

There is a simple workaround: Use system (plus exit, if necessary) instead of exec. While the semantics are slightly different from exec, they are at least the same on both platforms; system spawns a new process and then waits for it, returning its exit code. Notice that system has a list form too so you don't need to worry about shell escaping either.

SO_SNDTIMEO and SO_RCVTIMEO

nospam@example.com (Daniel Fischer) — Tue, 24 Mar 2009 13:30:29 +0100

Implementations of the BSD socket interface support various socket options. Two of them are SO_SNDTIMEO and SO_RCVTIMEO. They allow the user to specify a timeout for otherwise blocking send() and recv() calls. They're often described as the two socket options that have the most different implementations and are therefore among the most unportable ones at all.

It's not quite as bad, but among the major UNIX and Unix-like operating systems, there are at least three different beaviours for stream sockets, ignoring that the behaviour for datagram sockets may yet be different.

Most of the operating systems I tested this on actually support these timeouts. Mac OS X, Linux, FreeBSD and AIX were all among the platforms where I was able to use these timeouts in a simple test, using reasonably recent versions of these operating systems. This means that the OSes accepted setting the option and the timeout that was configured did actually work. The BSD socket implementation that is used in Microsoft Windows also falls into this category, the timeouts work reliably there.

One notable exception was Solaris, which reported that the protocol did not support this option. This means that the setsockopt() call failed. Since this is detectable, it's not a problem; there are other ways to implement timeouts without support from the socket layer.

The other notable exception was HP-UX (tested with 11iv3 and 11iv1). On HP-UX, you can get the same behaviour as on Solaris if you use the UNIX03 socket library, meaning setsockopt() fails with ENOPROTOOPT. However, if you're using the BSD socket library that is also provided with HP-UX, then you are allowed to set the timeouts, but they are silently ignored.

Even worse, the system will remember the timout you set, and querying it with getsockopt() will return it. This means you can't verify that the timeout is available by querying it after setting it and comparing it to the value you set it to. Setting it is allowed and querying it will yield the timeout that was previously set, but the timeout setting will be silently ignored by HP-UX.

In summary, you can actually expect these timeouts to either work or cause an error when you try to set them on most UNIX and Unix-like operating systems, and also on Windows, but if you are really concerned about portability, you need a backup plan, and either a whitelist of platforms where the backup plan is not necessary, or a very short blacklist that mostly consists of HP-UX.

UNIX domain sockets

nospam@example.com (Daniel Fischer) — Wed, 19 Dec 2007 12:05:00 +0100

Unlike sockets in other domains, sockets in the UNIX domain are visible in the file system. Because of this, they're sometimes confused with regular files. When they're not being confused with regular files, their specific restrictions that don't apply to other files are still easily forgotten. One such restriction of a UNIX domain socket is the length of its name.

Path names can be rather long on UNIX-like systems these days. Gone are the days of file names limited to 14 characters and for path names, POSIX-compliant operating systems generally support up to 256 characters. On many platforms, path names can be even longer than that. For example, PATH_MAX is 1024 on Mac OS X and 4096 on GNU/Linux.

In contrast, the full path name of a UNIX domain socket must fit into a struct sockaddr_un. Its component sun_path generally has much less room for the socket's name than 1024 characters. Typical numbers are 108 (Linux, Solaris, Cygwin), 104 (AIX, BSDs, Mac OS X), and 92 (HP-UX). At some point, it was only 14 in Interix 5.2.

This means that, while UNIX domain sockets do appear as if they were files, they can't be placed in an arbitrary location in the file system. Now, imagine an automated testing process that can test multiple instances of the software at a time, and keeps all files relevant to one test run within one directory. Path names can easily become longer than 92 characters in a scenario like this. In one case, this happened in a system where the name of one such instance's directory didn't have a constant length and occasionally brought the complete path to more than 92 characters, causing random total failures on HP-UX.