Icecast -kh branches

This is the current set of branches of ices2 and icecast2. They extend on the offical release (currently 2.3.2) with features that may be (if found to be working out well) merged into the next offical release. You can contact me karl <at> xiph.org

Main releases

23/05/2013

• processing certain mpeg intro file could lead to a listener disconnection.
• Auth remove could be triggered twice if no stream content comes in before time out
• Incorrect content type sent for aac files
• The lag check on the queue could trigger a crash, One case was handled now both are.
• Drop SO_REUSEADDR for win32, can break things over reload.
• Make file requests use normal HTTP response, while streams go through the agent checks. It seems that for asx playlists, wmp didn't handle ICY, but requires it for streams
• Fix up the log size setting when using the 2.3 way or doing it. it was being ignored
• Add ID back into per-listener details as some use 2.3.2 pages, typically centova users
• 64bit type clean up.

24/04/2013

• processing certain mpeg intro file could lead to a listener disconnection.
• initialise mpeg resync count on first frame found, prevents looping fallback files with ID3 or other content from dropping listener after so many times.
• fix relays not overriding listeners from a fallback when starting up.
• prevent the (n appearing in metadata for mp3/aac streams.
• allow for source queue shrinkage. With the rwlock merging, the queues were left to expand to queue-size whether listeners were lagging or not, this helps reduce unwanted parts of the queue.
• Avoid listener remove event if source client passes auth but is rejected due to the source being in use
• introduce lazy slave stats update for updating virtual mountpoints, this reduces the stats generated if you have many mountpoints with frequent starts and stops.
• drop PE substitution in metadata, only do the likes of &
• allow for listener processing to be skipped if it would stall waiting on lock.
• stats link update, like issuiing NEW commands first at connection before EVENT, not important for most people
• Various log message updates and small scale lock range updates.

18/02/2013

• update response header code for returning 200/206. On streams, do not send 206, but allow it for file requests.
• migrate some legacy source stats calls to newer API, reduces workload.
• fix some corner cases for queue handling.
• tweaks for response headers for iphone and blackberry
• fix for some cases where listener_remove is not triggered.
• fix for override fallback file through cascade.
• do not use spin locks on BSD for now, use mutexes.
• mpeg parsing for processing files handled better now.
• fix for handling privileged ports over reload.
• fix for SOCK_CLOEXEC on linux where it is not implemented
• allow for multiple args to be supplied to on-[dis]connect scripts.
• various performance tunings all over, along with a few race fixes and memory leaks

12/09/2012

• crash fix with intro files. intro reader requires larger buffer

2.3.3-kh4, Use kh5 instead

09/09/2012

• burst size of less than 1400 was not handled correctly with recent rwlock changes
• some memory reduction for FLV listeners.
• Missing xsl could cause a busy loop.
• fixed a crash when listener reached end of the queue. Certain assumptions about buffer of queue accounting were changed with rwlocks, exposed a crash case.
• on some multi-processor setups, it was possible that certain listeners could trigger a busy worker due to old (cached?) data, work around applied.
• A listener that started off a new on-demand relay could be only partially initialised a lead to a crash later on
• report of recent flash update on FF15 when requesting non-FLV content. Expected header identifying flash is no longer present so now we check for swf in referer.
• Add short write handler for clients requesting short ranges (eg Apple)
• The stats handle for normal file or fallback files were altered in a previous release which prevented a lookup of listener details via admin requests. now fixed.
• only move inactive relay client to another worker when there is sufficient difference
• honor auth options "allow_duplicate_users" and "drop_existing_listener" settings for webroot files
• proxy cache avoidance headers more generally used.
• minor memory leak on moving clients over fallback.
• code cleanup

07/08/2012

• crash fix with invalid stats handle with fallback to file
• merge Opus module from trunk
• Force NTPL to prefer writers over readers for rwlocks
• bitrate estimation scales better now.
• where possible, use pread to avoid mutex locks on file reads, win32 needs fixing up
• make xsl lookup cache scale better, force a reload via a separate thread to prevent worker stalls.
• various tunings for client rescheduling
• small changes to client balancing across workers
• small changes to log messages.
• reference a git hash code is available

15/07/2012

• sent value passed to listener_remove in kh1 could be wrong on 32bit setups
• avoid clearing queue on relay restart, it exposed a crash case with lock changes
• make TCP sockets not pass to on-[dis]connect scripts. useless work avoided
• hijack source fix required with lock changes, to avoid mis-calculated lags drops
• refcount bug missed for sending to master relays. possible memory corruption
• more worker balancing changes, to spread loading more evenly.
• missing delay for blocking file client, has an effect of higher cpu usage
• rare override failure case handled better.
• various minor fixes, including log format, memory leak and stats.

24/06/2012

• Big change with rwlocks replacing mutexes. It's internal but can have a big effect on load pattern usage. In cases with many listeners on a single mount we can now spread clients across all workers. A number of balancing changes were done to assist in localising related clients but may need more tweaking. A side effect of these changes is that queues are only shrunk when there are no listeners, so queues can take up more memory than usual but that could of happened before.
• fix bad reference for file-based content like intro/fallback with FLV
• make the minimum duration of an on-demand relay last until it would be classed as working, any less then it could be skipped or disabled for a short time.
• skip entity decoding if none required. For cases where metadata has & causing the libxml2 to spit out warnings like "no name".
• If no sockets are opened at the start then exit.
• Win32. standardize on format for date, back to proper timezone numbers
• Log AAC samplerate change in stream.
• Add <inactivity-timeout> in <limits> to stop icecast when there are no sources
• Add <x-forwarded-for> into <paths> for IP substitution from load balancers
• Added referer= to listener_add POST.
• Added sent= to listener_remove POST (bytes)
• sync ebml/webm format parser with 2.3.3, not tried it myself.
• minor tweaks for rescheduling on high bitrate streams, terminating sources, URL auth handling if disabled and busy relay restarting.

19/05/2012

• Expand file open limit to match <clients> if possible, warn if less.
• Fix possible client leak. legacy code removal.
• updated mpeg parser to be more specific on bits to match for. It should increase the tolerance for changes in the stream format (CRC, samplerate etc).
• reload forces an update of active streams, was missing from a previous update.
• Realplayer based apps fix for aac+. Assumes certain response header.
• drop requirement for matching each / in mount-name.
• content type needed truncating in rare cases before format could be set up
• prevent too much being read in one go when reading ogg data.
• internal change to swapping of hijacked clients, possible crash fix.
• fallback over multiple mounts works again
• Add timeout for sync mode handling. Should never trigger really.
• a few small log reporting changes.

17/04/2012

• Reduce memory copies with large stream lists from master
• internal FLV buffer management cleaned up, possible crash case avoided.
• return 501 instead of 400 for HEAD requests, helps some players.
• stats updating reduced now that relays always have clients, also reduce locking.
• missing user/pass setting for metadata update from shoutcast style source and url auth
• Experimental. send variable block framing if requested by listener
• Experimental. allow <include>/dir/*.xml</include> in <icecast> for breaking xml up
• Experimental. webm work by David Richards
• Experimental. source client hijacking. replace existing source with another. has to to allowed via auth url currently to prevent bouncing of clients with auto-reconnect.
• Add duration to all access, error and playlist logs, allows re-opening them after a number of seconds elapsed, 0 by default. set size trigger to 50Meg default.
• possible busy loop in kh31 in cases where fallback is defined
• fix for win32 error reporting and missing uninstall link in programs list
• added "timeout" (default 5) and "on_error_wait" (default 60) to auth url options.
• sync up headers handled by auth cmd with auth url.
• Added missing stream format checks for certain error cases.
• speedup for icy metadata and avoid length bug on inline urls.
• A number of possible lock fixes and checks are now added for odd cases.