Memory (RAM) Woes

[emoon]

First off, thanks for putting this tutorial together, Jeff. Great inspiration to get back in to hacking again.

Now, onto my problem

I've been doing a bunch of re-compiles to get my firmware setup exactly how I wanted it and finally, it was working great with the radio streams or a few mp3s.

However, when I samba mounted my entire MP3 collection to the 520gu from a file server, mpd threw up on me by running out of memory and took the system down with it (full system halt after 10s of memory allocation errors). The mp3 collection is about 65GB (i had a huge cd collection and am a big e-tree user) and that translates into an approx 2.3MB mpd.db file.

I have a few ideas, but don't really like any of them too much enough to do much more than do some experiments. Any feedback or other ideas?

Possible workarounds:

• Add a USB flash thumb drive and put swap space on it.
Pros: Easy and Cheap, Cons: worried about reliability of flash reliability as swap
• Create a small networked partition to use as swap space
Pros: Should be quick and easy, Cons: can swap space even be mounted over a network? Seems sketchy even if possible, especially wireless.
• Cut down memory usage of other parts on the system to minimal
Pros: no external swap needed, Cons: painful and really limits any improvements/expansions to the system.
• Buy the next step up of the router with 2x memory
Pros: probably would have enough to not have issues, Cons: Feels like giving up and throwing HW and $$ at a technical challenge :D

Am I overlooking anything? Has anyone else run into RAM limitations?

[jeroen94704]

I'd go for putting swapspace on a USB drive. That's what I do anyway. Yes, the flash memory will wear out eventually, but this won't happen quickly. In the meantime, it's very reliable.

For reference: There's an OpenWrt-derived distribution that I use (PacketProtector) which stores its swap-file on the USB drive, and experience shows a flash drive will last a couple of years in that setup.

You _could_ use a small 2.5" HDD as your USB-drive. That would take care of the flash memory wearing issue, but may not be an acceptable option for you (size, power, moving parts etc). If you do this though, make sure to connect it to a powered USB-hub, for these routers are not able to provide enough juice over the USB port.

Hope this helps,

Jeroen

[emoon]

Thanks, that is what I was leaning towards (usb stick as swap).

I need to change my thinking from 8hr/day heavy desktop usage to occasional-use music appliance... it'll would probably take a few years to get a failure in this application, if it ever happens.

[mightyohm]

I would do the USB swap thing. Components are cheap and replaceable, and this is all for fun anyway.

Let us know how it works out.

[ilinux]

Can you maybe try to put db file into samba share ?

[emoon]

Can you maybe try to put db file into samba share ?

The db is in the samba share, the problem is (discovered after a bit of research on mpd) that a 'feature' of mpd is that it loads the entire db into RAM. Normally this is a good thing, as it prevents needing to access the db on a drive and speeds up operation. However when you're dealing with 14MB or so of usable memory, 3-4MB is getting towards 1/3 of available.

So I did attach the USB thumb drive, and got it set up as swap space (512MB of swap! I feel like the richest man on earth after trying to squeeze it down to 14MB! ) However, as is often the case, I tweaked a few too many build options and... the network is kaput. So back to the beginning with a bit more knowledge and experience to do it all over again... that's what makes this fun, right?

[mightyohm]

Hey, at least you're making progress. Good luck.

[bubak]

I had the exact same problem.

The following was my almost-solution, which I have been using for several months. I run mpd on a real PC on the network (the one with the music library) instead of on the router. I set the mpd output as esound, and run only esd on the router. So the mpd on the PC streams to the router, and the router plays back the sound. The good thing is that the PC has lots of RAM for mpd, and esd on the router only needs very little. It works great except for one thing. Esound does not do any buffering, so if I use the wireless heavily on another machine (e.g. large downloads, but even graphics-heavy websites), the sound stutters. I put up with the stuttering for a while, but now it's getting annoying.

So now I'm looking for a better solution. Some ideas I'm thinking about:
- is there any remote audio server like esd, but which does some buffering?
- is it possible to hack mpd to not load the database into RAM?
- I could run an ethernet cable to the router, but that means tearing up the walls...
- a router with more memory, or some other hardware... suggestions?
- an 802.11n router, but would that help?
- mounting a usb stick as swap

Any suggestions?

[jeroen94704]

You could use the http or icecast output option of MPD on your PC, and run MPD on the router to log into and play the stream remotely. In my experiments, the http output of MPD works very well, and is very easy to set up, but is has one drawback: When playback ends, your clients will lose connection. Icecast has a fallback option to play a "silent" sound as a when there is no music to play. This ensures that connected clients will stay connected, even if you playlist runs out.

Jeroen