Smithology

Last week I spent a couple of days debugging some code I’d written that used sysv semaphores in linux. Semaphores come in handy if you want to model a situation where only one process can access a resource at a time. You can also use a lock file for similar ends but semaphores are a bit smarter.

Anyway, I was following the Linux Programmers Guide which is by no means comprehensive but does have a good section on IPC (that’s inter process communication) and found that I could use SEM_UNDO when getting a lock so that if the process holding the lock dies unexpectedly, its lock is released by the linux kernel. However, the gotcha here is that SEM_UNDO is just implemented as a counter so that when you do a lock with SEM_UNDO, the kernel just increments a counter (semadj) of how many adds it needs to do when the process quits. If like me you decide not to use SEM_UNDO when you also release a semaphore, then you’ll find that eventually the semadj counter will overflow, the program will crash and then linux will increment your semaphore by 32767! (This is a bad result since you’re typically using a semaphore that is valued at 1 to represent a free resource and 0 to represent the resource in use, a value of 32767 implies you have 32767 resources available: i.e. there won’t be much locking going on anymore).

The upshot of all this and what the documentation doesn’t make clear is that if you use SEM_UNDO when you do a semop to decrement the semaphore, you must also use SEM_UNDO when you increment it so that semadj doesn’t get out of whack. Fortunately, semctl called with SEM_SET resets the semadj counter (see ‘man semop’).

[tags]linux, programming, semaphores, sysv[/tags]

Linux

Identi.ca is a twitter clone that runs on open source software called laconi.ca and will apparently be distributed instead of tied to the one host. More like the original blogging idea.

I’m checking out how locani.ca works by downloading and attempting to run it. I should be able to somehow tie my “tweets” back into the main identi.ca site.

Keeping an eye on this Evan who seems to be the brainz behind the operation.

So far, I’ve had to go to the UNOFFICIAL google apps mirror since laconi.ca is down. I had to get the code using svn.

The code is written in PHP which may or may not be a good thing depending on your specific religion.

Actually I just discovered you can get into the laconi.ca site if you know a deep link e.g. http://laconi.ca/Main/Source

To install the required pear stuff (see the tarball/docs/README):

mkdir pear
pear config-create /home/mypath/laconica/pear/ pear.config
pear -c pear.config install validate
# which told me to run the following instead because it is beta
pear -c pear.config install channel://pear.php.net/validate-0.8.1
pear -c pear.config install db_dataobject
pear -c pear.config install Mail
pear -c pear.config install Net_SMTP

wget http://xmpphp.googlecode.com/files/xmpphp-0.1beta-r21.tar.gz
tar zxvf xmpphp-0.1beta-r21.tar.gz
less xmpphp/README

wget http://openidenabled.com/files/php-openid/packages/php-openid-1.2.3.tar.bz2
tar jxvf php-openid-1.2.3.tar.bz2

surf to http://testserver/php-openid-1.2.3/examples/detect.php and everything is ok.

wget http://oauth.googlecode.com/svn/code/php/OAuth.php

wget http://michelf.com/docs/projets/php-markdown-1.0.1m.zip
unzip php-markdown-1.0.1m.zip
mv PHP Markdown 1.0.1m/ markdown-1.0.1m

Because I’m using a nonstandard PEAR path, I had to edit my extra_path thing and then copy the two lines (extra_path and set_include_path) to the top of lib/common.php as well.

$extra_path = array("/home/mypath/laconica/php-openid-2.0.1",
 "/usr/local/share/php",
 "/home/mypath/laconica/pear/pear/php",
 "/home/mypath/laconica/pear/pear/lib",
 "/home/mypath/laconica/markdown-1.0.1m",
 "/home/mypath/laconica",
 "/home/mypath/laconica/xmpphp");
set_include_path(implode(PATH_SEPARATOR, $extra_path) . PATH_SEPARATOR . get_inc
lude_path());

While you’re at it, you should set the ‘server’ and ‘path’ config options because otherwise you will be getting redirected to localhost all the time (which doesn’t work so well if you’re not running the browser on the web server).

<b>Fatal error</b>:  Call to undefined method XMLWriter::fullEndElement() in <b>/home/matthewsmith/public_html/laconica/darcs/lib/util.php</b> on line <b>104</b><br />

I need decafbad to come back online so that I can se how he got past this error.

more soon.

[tags]identi.ca, laconi.ca, microblogging, twitter[/tags]

Linux

Unix domain sockets can be a bit fiddly. Here are some mistakes I made:

• for datagrams, use recvfrom and sendto. There is no need to do the listen, accept or connect calls for datagrams. Just bind().
• make sure when you call recvfrom, you initialise the fromlen value to be the size of the sockaddr_un struct otherwise it will get confused as to how much space it has to populate the struct and weird things might will happen.
• When using the unix abstract address space (rather than the filesystem), the first byte of sun_path should be a ‘\0’. But you need to get the length of the string before you clear the first char so that you can pass the true size of the struct to the bind() call. See ‘man 7 unix’.
• I found this example code very helpful (although it uses the filesystem, not the abstract address thing).

This is a short post but it represents more than two days of insanity.

[tags]ipc, linux, programming, unix sockets[/tags]

Linux

Did I say the other day that I like my Dell D630? Was I mentioning the awesome power of the dual core intel and the 2 gigs of RAM after using my bottom of the range G4 iBook.

Well I have one of two quibbles, the networking on this thing has made me more angry and caused me to swear more than anything I can think of. The wireless does seem a bit better since I upgraded the Intel ProSet thing. Like now it doesn’t just randomly drop out and then decide it can’t connect to anything unless I reboot.

I just spent the last two hours (which was going to be some nice relaxing DVD time) battling it out with the fricking piece of shite internal network card. I need an internal network card to be able to copy stuff off a certain computer at home – actually I can do it wirelessly but the network is way faster for certain large files.

The Dell D630 has a way cool feature that disables the internal network card when it is running from batteries in order to save power. Way cool until you actually want to use the network port and it stays disabled.

Things I tried:

• unplugging and plugging the cable multiple times
• rebooting
• using the Dell quickset to ‘Always activate on battery’
• turning the ‘Always activate on battery’ with various combinations of plugged and unplugged cables
• rebooting with cable plugged in
• rebooting with cable unplugged
• plugging into mains
• doing all of above combos whilst plugged in
• reinstalling drivers
• swearing
• unplugging and replugging power, network in various combinations
• cursing
• swearing
• power cycling the network switch
• attempting to enable the device from the device manager

All this time, the little lights on the network port and blinking in time to the pings from my linux box. What the hell is going on? Oh yeah and I should mention that at some point the device disappeared from the device manager. Then it came back after a reboot with ‘Device is disabled (code 22)’ and an enable button – which I pressed. A few times. Then I swore at it. Then I tried it again. Then I tried it with the cable unplugged, replugged, etc…

Then I just went into device manage and deleted the device. Rebooted and as it was rebooting: “64 bytes from 192.168.1.20: icmp_seq=7859 ttl 128 time=0.371 ms”. Yes, Jesus thankyou at last! Don’t ask me to explain – I suppose it managed to find the driver and reinstall it as it was booting.

No thanks to you Dell, Broadcom and anyone else responsible for this debacle. I’m sure it was the sales guy’s fault. It always is.

UPDATE: The plot thickens, it might not be the laptop’s fault at all. It might be something to do with my switch or the linux box’s network card. If I ever get the time, I will Wireshark it and get to the bottom of this madness!

UPDATE: It is definitely the Dell as evidenced by the number of people commenting here. It seems uninstalling quickset helps.

[tags]broadcom, dell, network[/tags]

Linux, Uncategorized

I couldn’t find instructions out there so here are my notes:

downloaded from svn

svn checkout svn://svn.lighttpd.net/lighttpd/trunk/
mv trunk/ lighttpd-1.5-svn-2119
cd lighttpd-1.5-svn-2119

Mucked about with the auto tools

libtoolize
aclocal
autoheader
automake
autoconf
autoreconf
automake --add-missing
autoreconf

got error:

./configure: line 25224: syntax error near unexpected token `XML,'
./configure: line 25224: ` PKG_CHECK_MODULES(XML, libxml-2.0,'
make: *** [lighttpd_source] Error 2

Realised I needed to use cmake

sudo apt-get install cmake
cd lighttpd-1.5-svn-2119
cmake -L -DCMAKE_INSTALL_PREFIX:PATH=/usr \
         -DWITH_PCRE:BOOL=OFF .

Sadly I pretty quickly get an error to do with a program called lemon. It appears that lemon is a helper app that is compiled during the make but due to the cross compile, it can’t run.

I was able to get around this by manually compiling lemon first in the host format and then cross compiling the rest of lighttpd

cd lighttpd-1.5-svn-2119
CC=/usr/bin/gcc cmake -L \
    -DCMAKE_INSTALL_PREFIX:PATH=/usr \
    -DWITH_PCRE:BOOL=OFF .
make lemon

Now do the cross compile config

 rm CMakeCache.txt
 CC=/opt/eldk41/usr/bin/ppc_6xx-gcc \
    LD=/opt/eldk41/usr/bin/ppc_6xx-gcc-ld \
    cmake -L -DCMAKE_INSTALL_PREFIX:PATH=/usr \
    -DWITH_PCRE:BOOL=OFF .

finally touch ‘lemon’ so that it won’t be remade

 touch build/lemon
 make

Hmm. This blog theme is not the best for displaying code.

UPDATE: once I got it to cross compile, the other differences were that I had to add a new required module mod_chunked.so and also a dependency on librt.so.1 (I don’t automatically copy all of the eldk compiled libs to the target unless i need them)

[tags]cross compile, eldk, lighttpd, linux[/tags]

Linux

Here is my solution to the problem of Premature end of script headers: awstats.pl (which sounds like something you might need nasal spray for doesn’t it?). This seems to be an issue for systems using virtualmin but maybe other web hosts can have trouble with running cgi but will allow php.

File: awstats.php

<?php

$cmd = "export QUERY_STRING=\"{$_SERVER['QUERY_STRING']}\"; "
        ."export GATEWAY_INTERFACE=\"CGI/1.1\"; "
        ."export HTTP_USER_AGENT=\"{$_SERVER['HTTP_USER_AGENT']}\"; "
        ."export SCRIPT_NAME=\"{$_SERVER['SCRIPT_FILENAME']}\"; "
        ."/usr/lib/cgi-bin/awstats.pl";
$res = `$cmd`;
$res = str_replace('awstats.pl','awstats.php', $res);
$res = substr($res, strpos($res, '<!DOCTYPE'));
echo $res;
?>

It’s a wrapper script that calls awstats itself and then passes the results back to the browser. Note it clobbers any headers awstats sends back and replaces references to awstats.pl to awstats.php in a rather brute force way.

[tags]awstats, cgi, web hosting, wrapper[/tags]

Linux

A little while ago our VCR broke and I decided that I might try out this new PVR craze. I have an old Pentium III 550MHz with 256MB RAM running Gentoo and it has been functioning as a personal webserver for the last few years.

I’ve taken some notes here so that I can remember what I did and also to add any gotchas that I came across on my particular system.

The first thing I needed to do was get the PC on the wireless LAN so that I could hide it behind the TV and not have a blue cable running all over the house. I had a Linksys DWL-122 USB wireless adapter handy. With a little googling I discovered it is a prism2 chipset which means it needs the linux-wlan-ng drivers. I followed this guide for gentoo and got the drivers going. Getting the connection to come up at boot was not straight forward. I added the prism2_usb to /etc/modules.autoload/kernel-2.6 for starters. I added the gateway and IP address settings to /etc/conf.d/net. That was easy. I tried a few times to get the init script to work but gave up and instead copied the net.eth0 init script to net.wlan0 and modified it so that the iface_start function looks like this:

iface_start() {
modprobe prism2_usb prism2_doreset=1
wlanctl-ng wlan0 lnxreq_ifstate ifstate=enable
wlanctl-ng wlan0 lnxreq_ifstate ifstate=enable
wlanctl-ng wlan0 lnxreq_autojoin ssid="MySSID"
authtype=opensystem

Those commands should hook up the wireless stuff before the network configuration gets going. The first command runs twice because sometimes it fails. I also find that on the occasional reboot, the USB device gets in a weird state and won’t come good (even with reboots) until I unplug it. It was a bit of a time waster because I was booting new kernels at the time and thought that the new settings had created a conflict.

I chose the Hauppauge PVR 350 because it has onboard mpeg2 encoding and decoding which I thought would help my old CPU to cope. The Pentium III plays DivX ok but sometimes gets choppy. I also bought a big HD (Seagate IDE 320GB) as I was running out of room. Once I had those installed, I found this guide for MythTV and Gentoo which I followed closely with a few excursions off the side as documented below.

When configuring the kernel I also needed to setup the IVTV drivers. I followed this guide. The IVTV drivers require a whole bunch of stuff to be turned on in the kernel so that they work. I use ‘make menuconfig’ and it’s not always obvious which menu items the kernel defines map to but you can check them by choosing the ‘help’ button in menuconfig.

In addition to the ones in the MythTV guide, I chose the following options:

Multimedia devices  --->
  <M> Video For Linux
    [ ]   Enable Video For Linux API 1 (DEPRECATED)
    [*]   Enable Video For Linux API 1 compatible Layer
     Video Capture Adapters  --->
        <M> Conexant 2388x (bt878 successor) support
        Encoders/decoders and other helper chips  --->
            <M> Micronas MSP34xx audio decoders
            <M> Wolfson Microelectronics WM8775 audio ADC with input mixer
            <M> Philips SAA7113/4/5 video decoders
            <M> Conexant CX2584x audio/video decoders
            <M> Conexant CX2341x MPEG encoders
            <M> Philips SAA7127/9 digital video encoders
Graphics support  --->
  <*> Support for frame buffer devices
  [*]   VESA VGA graphics support
  <*>   Trident support

For a window manager, I already had TWM compiled but I’m not sure this is the best choice. The title bar doesn’t disappear when MythTV is running and I get weirdness when using the IVTV mpeg2 output encoding which I suspect is due to TWM.

The lircd setup went smoothly however, I found that the second part where you map remote control buttons to MythTV didn’t work. I chose to use a ~/.mythtv/lircrc file but the one I downloaded only mapped a few buttons. My remote is the grey and black type and it has seperate channel buttons to the arrow buttons. Here is my lircrc file with better mappings.

The cool thing about the IR receiver is that it’s on a wire so you can hide the computer and just have the IR receiver poking out front somewhere. It is quite small and black.

For XMLTV I used tv_grab_au_reg which is a xmltv grabber for Australia and signed up to the OzTiVo wiki which is a requirement for dowloading the data. There are instructions on the tv_grab_au_reg page to get it going with MythTV. I found that I had to add all my channels to the .xmltv config file like this:

<channel display="ABC" tvguide="ABC-Qld"/>
<channel display="7" tvguide="Seven-Bris"/>
<channel display="NINE" tvguide="Nine-Qld"/>
<channel display="Ten" tvguide="Ten-Qld"/>
<channel display="SBS" tvguide="SBS-Qld"/>
<channel display="Briz 31" tvguide="BRIZ"/>

Initially when I added the channels to MythTV I did an autoscan. I had to do it a few times using the back button to get all the channels. Doubling up here didn’t seem to matter. However when I went to mythfilldatabase, it added new channels from the XMLTV file into MythTV. To get around this, I deleted all my channels, did the scan again and then edited each one so that it had the correct callsign and XMLTVID (the callsigns are ABC,SEVEN,NINE,TEN,SBS and BRI31). That way when I ran mythfilldatabase it didn’t add new channels but did the right thing. I still have database entries for programs that don’t link back to channels since I deleted the channels (which means programs are listed twice on some screens) but I expect this will go away once I move past that data.

For TV Output, I followed the instructions for PAL IVTV on Gentoo. These were pretty spot on but failed to mention that your need to set TV in your ServerLayout in xorg.conf and also modprobe the ivtv-fb module. As soon as you modprobe that module, the output of console will go to TV out of the PVR card which results in a helpful blank looking screen on your monitor and may cause a bit of panic. Anyway, once you modprobe that, the /proc/fb thing starts working. I added ivtv-fb to /etc/modules.autoload/kernel-2.6 too.

Now I am at the stage of learning to use MythTV. I’ve already recorded some stuff and watched a bit of teev. It is all pretty awesome. I’m still not 100% on the TV out thing – I had to make a setting to the offset so I can see one edge of the image. I have no trouble playing back recordings but the video can get very choppy when I’m watching live TV at times. I am going to try and get the whole thing going in framebuffer mode without X and see if I can get the onboard mpeg2 output decoding going too. That should give better performance. If I get around to it, I’ll post some more info about it on this blog.

UPDATE: See some photos here. I have installed a heap of myth plugins. They are pretty straightforward. Mythweather and mythnews didn’t do much. Mythmusic and mythgallery work pretty well – mythgallery is not so good on an iPhoto gallery as iPhoto tucks everything into little subdirectories everywhere.

[tags]gentoo, hauppauge, lirc, mythtv, pvr, pvr 350, tv, tv out, tv_grab_au, xmltv[/tags]

Linux

I recently replaced samba on my home server (i.e. old desktop) with WebDAV for use with my Mac. I was having issues with the samba connection dying. Part of this was that I was automounting the samba drives using the Netinfo Manager which meant that I couldn’t unmount and remount the usual way.

I found this post which outlined a more usable automount on the Mac using AppleScript.

As for WebDAV, I know there are issues with Windows but for windows, I’ve left the Samba share going. I decided to give it a go as I thought the Mac support might be a bit better. So far it’s been no problem. My home server is on Gentoo, so I used this guide

I have found that doing the username:password@server thing in the URL didn’t work (for me). I also didn’t have much luck with SSL which might have been something to do with my self-signed expired certificate with non-matching hostname :-)

Linux, Mac

Last Thursaday, I ditched Telstra mobile after seven years. The story goes like this: I made a lot of calls in April because I was buying a house and the loan application got a bit tricky. I had to make lots of phone calls, I had to be available so I used my mobile phone a lot. The bill came in at $180 for the month. I rang Telstra to let them know that I had had a bad month and maybe they could help me out – so the kid on the phone in their billing department flicked through her three ring binder (you know the one with all the scripts of how to field calls) and offered to extend the deadline for the bill by two weeks. I asked her about changing plans to try and stop this happening in the future and she said I could go on a $49 cap but also paying $27 for the phone on top of that.

The next day I did some sums. My coworker is on a $29 cap with 3 mobile and has to pay $10 for his shit-hot Nokia E65 (ok so it’s not the best phone you can get but compared to the $27 I pay for the V3x). So over two years, I could go on Telstra’s cap and spend $1824 or I could cop a $350 fee to get out of my contract and pay $960 to 3 and get a better phone, not including that I could sell the V3x (it’s not locked to Telstra’s network).

The only bit I got wrong was that in my enthusiasm, I told Telstra to stick it before I’d talked to 3 so I couldn’t transfer my number.

The really good bit though is that the E65 is blowing my socks off. I installed Fring on it today and skyped out on my home wireless network. Yesterday, I was in Sydney and I surfed the net for free in the Qantas lounge. Very nice. The wireless is very cool and it seems like there is a healthy amount of development of good stuff for Symbian. My coworker has given me heaps of good advice to pimp it up so I’m very happy and not too sad about having to cop the steaming turd of a bill that I’ll get from Telstra next month.

Here is a little transcript of some links from my coworker.

3/05/2007 3:34:14 PM: http://www.businesssoftware.nokia.com/bc_search.php
3/05/2007 3:34:25 PM: best thing to get is this: http://www.businesssoftware.nokia.com/mail_for_exchange_downloads.php
3/05/2007 3:34:53 PM: look here for software you might like (eg tomtom) http://www.ipmart-forum.com/index.php
3/05/2007 3:35:34 PM: and congrats you now have a symbian phone: http://www.allaboutsymbian.com/
3/05/2007 3:37:43 PM: see my latest update on page 10: http://forums.whirlpool.net.au/forum-replies.cfm?t=715434&p=10
3/05/2007 4:22:07 PM: this apparenlty gets around the problem of some apps not supporting access groups: http://www.smartroaming.com/public/index.php?mid=1
3/05/2007 4:24:13 PM: “I use birdstep smartroaming to do wifi with M4E. You can define an access group, but unlike the built-in method it seems to the Nokia that it’s a single point so it works in apps like M4E which seem to require a single, rather than group AP.”
3/05/2007 4:25:31 PM: how to setup vpn policy for it: http://discussion.forum.nokia.com/forum/showthread.php?t=80837&page=12
3/05/2007 4:26:32 PM: and apparently the smart roaming app can auto connect to vpn as a trigger
3/05/2007 4:26:32 PM: nice

3/05/2007 4:27:29 PM: you might also want to use it as a voip phone 3/05/2007 4:27:42 PM: but ill let you figure that out 3/05/2007 4:27:52 PM: i have tried ‘fring’ – with its skype 3/05/2007 4:27:59 PM: but pure voip is much better apparently 3/05/2007 4:29:21 PM: heres a good vpn guide for the e65: http://www.e-series.org/archives/246

The only thing I’m unhappy with is the non-existent OS X support.

[tags]3Mobile, E65, Fring, Mobile Phones, Nokia, Skype, Symbian, Telstra, WiFi[/tags]

Linux

I was trying to add dovecot to my virtualmin setup on slicehost on the weekend. The problem was that virtualmin wrongly detected the version dovecot and wrote the wrong config file. So then I couldn’t get dovecot to start. I ended up following a guide on setting up dovecot to work with ubuntu and postfix using PAM - once I set the userdb and passdb and the sasl config options it all came good. It was pretty straightforward and I can’t really remember the link I used so sorry about not being more helpful – actually it might have been on the dovecot wiki.

To test it I was using mutt -f imap://username.domainuser@localhost/ which started working once I figured out to configure PAM in the dovecot config. However, I still had trouble with pop3. This ended up being because of a thing called uidl which I hadn’t set.

Next step is getting IMP onto the thing. I would like to check out squirrelmail as well to see how it’s looking these days. Does anyone know of any cool rails webmail clients? A gmail knockoff would be cool.

[tags]dovecot, slicehost, imap, pop3, mail, postfix, sasl, virtualmin[/tags]

Linux