Icecast -kh branches

This is the current set of branches of ices2 and icecast2. They are based on the Xiph offical release with features based on feedback from people. Some of these features are not suitable for distribution from Xiph, like the FLV wrapping/mpeg parsing, others are more suited for large scale installations. You can contact me karl <at> kheyes.plus.com

Main releases

7/3/2023

• config lock imbalance fix. Shows as a lockup on reload or exit.
• request HTTP 1.0 on relay requests, some send chunked responses which make a mess of ICY metadata.
• other minor code changes that do not have a functional effect.

5/3/2023

• Another update to xml parsing. Allows for cleaner and more flexible handling of parameters and tags. Added fallback to mirror fallback-mount, so that it reads better in certain cases eg <fallback mount="/backup" rate="128k"/>. The other layouts are also handled. Empty tags are now ignored, seen with some with intro tags original shoutcast port compatability settings handled better.
• Optional feature, set hijack to true on mount to allow a source to kick off the existing stream without touching listeners. This is primarily for replacing a relay stream as a replacment for fallback handling as it keeps listeners on the same mountpoint. While the feature will stay, the exact methos to enable is still subject for change but setting this on the mount block is reasonable.
• fix relay intialization failure case.
• relay internal state fix on reload. restart on certain changes.
• range header case fixed. playback issue in browsers
• various code cleanups, possible lockup issues resolved, crash cases, work done at shutdown etc.
• reduce some excessive logging in certain cases.

2/2/2023

• Extend http headers work from previous release. Can now be used to create POST data. This improves code readability and also allows for more flexability. Make global http headers work as expected.
• Switchover updates. Cleaner handling when switching over the client for a relay or source. Interacts with inactive on-demand relays.
• Introduce the linger-on mount setting. On stream exit, the stream goes into a state where it stays around running but no feed comes in. This allows for a delayed termination if you have relays or sources that drop but reconnect but you do not want to keep dropping to fallbacks. During this state, you can connect another source so can be used as a switch over between sources.
• Various libcurl interaction updates. tigher control on callback handling Handle server shutdown interrupting libcurl requests quicker.
• Trap for HTTP request on shoutcast port and treat it as such.
• Various code cleanups that have been reported.

18/1/2023

Report on the global http-headers causing a crash, now fixed in master, will update these build soon but you can use these if you do not intend to use global http-headers.

• HTTP handling update. A lot of this is internal but the mechanism allows for updating http headers from the XML using the http-headers block at the global level or per mount point. The same sort of block can be used within the relay for specifying any headers to a relay.
• XML parsing updates. This allows for simple tags to be embedded into the parent tag. examples include
  

      <listen-socket port="8000" />
      <mount mount-name="/stream">
      ...
      </mount>
      <relay local-mount="/stream" on-demand="true">
        <host server="master1" port="8000" mount="/feed-1" />
        <host server="master2" port="9000" mount="/feed-2" />
      </relay>
      

• Relay hijack capability. Two possible use cases
  • In the multiple host relay situation, every relay-delay seconds, the previously failed but higher priority hosts are re-tried and if one works then switches in to provide the feed for the stream. listeners are not moved.
  • A source client can override a relay on the same mount point without moving listeners. This effectively shuts down the pulling part of the relay but stays around until the source client exits, at which point the relay restarts. This can be seen as another type of fallback, one without the issue of shared fallback mounts causing listeners to end up on unexpected mount points. As the listeners stay in one place then things like stats are more consistent.
  
  
• SSL can be used on relay feed. Use boolean tag ssl or tls on the above host. The relay can redirect to https to achieve the same thing.
• priority tag on relay to allow for order host startup different to the order in the XML. 1 is highest order, unspecified implies ordered per XML.
• add ignore case for user agent matching.
• priority tag on mount to allow for search order when you can match multiple mount blocks (due to wildcard).
• Small update to CORS handling, interaction with http work.
• fnmatch (wildcard handling was broken in windows now fixed) and can use extended settings like @ etc
• Minor web page update
• Document updates.

7/12/2022

• Locking updates. Mostly for the config, mount lookup and client processing.
• Add redirect_limit to auth URL options, default 1 but 0-10 allowed.
• If URL Post is too large then avoid the backend request.
• Fix up the slow SSL response on web requests.
• The id flag on killclient command did not handle very large numbers
• Drop the source limit check on relay startup. Did not make much sense.
• Fix some keepalive client issues.
• log reopening fix
• Ogg FLaC header fix
• Various internal code changes.
• update auto* build setup

1/7/2020

• Locking updates. For most, it will be no different but some platforms can be affected
• Keep consistency on the global listener count. Could of shown as very large number.
• Fix issue with fallback override of listeners. listeners being dropped
• Fix broken parameter to on-[dis]connect script on previous release.
• Small update for CORS handling.
• Fix YP updating. Long standing error flagged up on directory changes.
• Update for listeners moving between workers for load balancing.
• Small update for the FLV-aac wrapping in previous release.
• Fix up range request bug.
• Fix various log related issues.
• Some internal code cleanup

26/4/2020

• Allow for using secs (eg 5s) in queue/min-queue/burst measures.
• Allow multiple icy sources on same port. Just embed in password a mountpoint:pass. The shoutcast-mount tag still creates the +1 port and sets a default mountpoint but source can override this.
• Fix log issue which could cause a deadlock if on-[dis]connect or auth cmd are used.
• Allow configure to accept ICY_CHARSET to set alternate in build. The default ICY metadata is assumed to be UTF8, but some sites still have issues with setting tags or parameters, so allow a different default if they need it.
• Regression on FLV wrapped aac metadata fixed.
• Crash/corruption fix if using stream auth and metadata updates via admin.
• Fix for glibc rwlock priority
• Added the icy-metadata in headers allowed with CORS
• Rework XFF to be earlier, useful in cases where banned IPs apply.
• Allow wildcards in XFF
• Drop the BSD NOPUSH setting for now. Have seen poor performance because of it in certain cases
• Openssl API cleanups. eg API differences between versions.
• various small fixes, build and operational.

28/1/2020

• Fix an annoying race case. memory corruption in certain cases that are hard to trigger normally but typically involes a listener disconnecting in certain conditions.
• Tightened up use-after-free cases with regard to the queue pruning.
• leak fixes with intro via auth.
• Allow seconds to be specified in burst and queue size fields. Just specify the number followed by s eg 30s or 5s. Eventually we could make this the default so that is can apply to various bitrates.
• small minor cleanups across the board.

26/3/2019

• fix up some incorrect or stale block cases from previous release for non-ogg streams. Usually cause by short write triggers or queue jumps.
• change default charset for non-ogg streams to utf8 instead of latin1, you ca still use latin1 either with <charset> mount setting or charset= query param to metadata
• global listeners count could get messed up with fallback to files, also make it a regular update stat in-line with other global stats.
• avoid possible crash case on log re-opening, not so common.

4/2/2019

• fix issue with timelimited listeners and html5 players with range requests
• security fix for auth url when allowing certain headers to be passed.
• fix for mpeg block expansion handling, led to cases of parsing failures
• block scaling changes for mpeg based streams.
• internal restructure for metadata on non-ogg streams. For code maintainability.
• bounce admin mount request clients to worker with the source, reduces lock contention with lots of sources

27/5/2018

• stop ID3 in streams for now, some browsers can have playback problems.
• fix memory leak case in the queue from previous release.
• fix crash bug on source timeout
• various internal changes for building, rate calculations

18/3/2018

• pre-roll logging options. like access log but report client details of intros played. Can be per-mount or global. Also allow for IP recording to prevent intro play if reconnected within N seconds.
• add tag ssl-cafile to refer to a CA bundle other than the default. This way the cert is not required to include it.
• allow /admin/metadata to accept param intro or preroll to change the intro file used on a source.
• split explicit mounts from list into a tree for quick lookup. speeds up updating with many mount definitions.
• max-listeners setting in limits for server wide limitations.
• worker balancing updates. avoid a lot of bouncing between workers for sources and allow more listeners to be attracted to their source worker.
• clean up shutdown procedure for race handling.
• smaller SSL handling cleanup.
• log messaging update.
• mpeg parsing updates. Allows for better detection and reporting, identifies mismatches better, and detects a few more frame types. The sync structure is also reduced so should help in caching.
• rate averaging update.
• minor tuning for intro drops, fix for 0- range requests and intros.
• delay opening intro file until stream has started up, allows for format matching.
• lessen write lock scope when reading xml.
• failing to bind a socket at startup should be a termination event, although not on a reload.

5/12/2017

• Logging. An issue was reported on log cycling triggers, should be the last now. This time it seems to stem from a cycle triggered externally where a new log is created for appending before doing the reopen. This is now fixed.
• Expand the worker limiter when processing lots of clients in one pass. This was leading to some seeing sources not getting cleaned up as the listeners were not getting serviced. There is still a cap but is extermely large and can be bypassed on cleanup cases.
• Incoming worker thread added. Normally incoming clients, after passing the initial IP checks get passed on to worker threads for subsequent pocessing. This can lead to a lot of wakeups for the usual worker threads to process these especially if they are short (eg playlist) or involve filtering or kicking off auth. The extreme case being a DoS type of thing, where the wakeup often causes the worker to do a big list traversal slowing things down as it gets busy. This new worker thread deals with the initial aspects, http parsing, initial request handling and is only moved to the normal workers for sources, listeners and some static content.
• Add "headers" and "header_prefix" options onto auth url. To bring into line with the Xiph build. It is possible that some sites would want to inspect certain headers passed by clients to authenticate. We do not want to pass all of them as that can be large and for such situations, particular headers are limited, the prime example of these is cookies. To avoid POST tag name clashes with existing established tags passed in aurl auth, these headers are supplied with a prefix, defaulting to "ClientHeader-"
• Stats. A number of global stats are now reported every 10 seconds or the next second if they have changed. This reduces load on a stats link and can save on logging if you have that level enabled.
• A small set of code cleanups

9/10/2017

• fix up recent ssl update to build against openssl 1.1
• drop range requests on streams, was breaking iOS 11.
• log typo fix

18/9/2017

• drop client is read channel is closed off, handles proxies better like AWS.
• Logging
  • better log messaging at startup if change owner used.
  • fixed race bug generally affecting the per-mount logs, excessive purging.
  • fix auto log cyclying
• Add listeners= to POST on auth url. current listener count in the stats
• internal improvement to cached entries for addresses
• SSL updates
  • build fix where no SSL is available
  • allow for DH/ECDH
  • allow for reading the cert before dropping privs, but keep certificate if reload is fails.
  • crash fix if no SSL certificate defined.
  • small fix for internal openssl threading.

8/5/2017

• use a separate thread for logging, prevents worker stalls in certain setups.
• backward compatibile playlist archive setting fix.
• update IPv6 handling. This is really an internal update but allows for both IPv4/IPv6 listening sockets to be created from one listen-socket block. This allows for windows builds to have IPv6 as well.
• autodetect SSL connections on incoming sockets. No need for <ssl> in listen-socket now but is still there for compatability.
• add <ssl-private-key> in <paths> to allow for combined PEM or for separate SSL key/certificate files.
• select https/http URL in autogenerated m3u based on incoming request.
• minor update to windows service creation. No functional change

17/01/2017

• handle really short icy intervals from relayed streams.
• updates for header handling, ranges, auth/html5 etc
• invalidate requests on windows ending with dot, could be used for extension trickery
• have clients stats reported once per second instead of after a change.
• use CORK/PUSH setting if available else fall back to nodelay.
• added so-mss setting to listen-socket.
• missing run-on setting for master now fixed, defaults to 30s.
• auth cmd drops LD_PRELOAD, stdin/out/err and handle errors better.
• various internal code cleanups, boundary checks, log messages
• build updates - autofoo mostly from rubenk, type cleanups for win64

28/10/2015

• fixes for range handling for certain players like iphone/chrome
• fixes for burst handling with intro playback
• ID3 pushed into the frame parser.
• adjustments to having a wider range of block sizes on non-ogg streams
• adjustments to scheduling of clients.
• added agent to listener remove on url auth
• force SSLv3 off the ciphers.

21/09/2015

• possible global listener count error fixed
• possible crash with FLV and fallback handling fixed
• FLV throttle fixup when dealing with files
• fix up case for requeue on lagging client.
• use throttling thresholds based on bitrate
• scale non-ogg blocks based on bitrate but still enforce a send limit with an optional <max-send-size> tag. not usually needed
• implementation of HTTP keep-alive, mainly for web type requests.
• add <redirect-to> tag in <mount>, similar to <redirect> but allows for expansion such as ${mount}. the redirect tag is just an alias of this now.
• intro file sending had no burst calculations.
• allow for auth bypass in certain very limited cases, like small range requests used by certain clients, or HEAD requests.
• HTTP header cleanups depending on the request, content length etc
• allow for byte accurate drop, this is for range requests for some clients.
• shutdown cleanups, to avoid stalls.
• tidy up xml page error reporting.
• build updates

04/05/2015

• fix for the 'Detected an entity reference loop' message
• header parsing in auth url updated to handle spacing better
• preload default mime types then try to load the mime types file
• fix ssl routines to handle stalls better
• fix issue raised by Klaas Jan, handle blank usernames and encode them to keep the validity of the CLF format.
• cleanup xslt cache/reader processing.
• make PUT the same as SOURCE, handle 100 Continue header
• allow type=.fla as a trigger for FLV wrapping
• user the real/effective/saved user/group calls if available
• prevent worker pipes being passed to exec'd processes, and force FD 0 1 2 to close
• fix header line Content-Disposition
• avoid possible race case in relay removal.
• for ogg feeds, make sure a new block is allocated at the start, else bad parsing could occur.
• use 2.4.x setting for default SSL cipher settings.
• allow for moving a lagging listener to the front of the queue
• send a CORS template for both HEAD and OPTIONS requests, HEAD needs expanding though
• open up the write throttle for unparsed file content
• use original mountpoint in listener disconnection for auth/accesslog selection
• Apply x-forwarded-for to source clients
• treat some failure cases correctly if auth presume innocent setting is in use
• add YP sanity checks for obviously incorrect cases
• various log message cleanups.
• various crash cases fixed.

04/09/2014

• repackage win32 installer to include a library dependency and theora capability.

02/09/2014

• prevent memory leak with short send xsl requests.
• avoid rare case of crash with delayed worker, seen with AIS but any substantial delay could of caused a race on stats with restarting sources.
• Add ID3 skipping to file reader. This prevents resync occurring.
• Added CORS headers for html based playback cases.
• added IP and user agent to POST for mount_add/_remove request
• relay run-on setting added (default 30 seconds). For on-demand relays, this is how long the relay will continue with no listeners.
• added message for excessively long duration spent going through a worker client list.
• for cached lists like banned IPs, place wildcards separately to fully specified entries, as the matching broke in such cases. The wildcards list now separate from the tree.
• add the nodelay thing back in, normally if doesn't make a difference but in certain cases it can affect packet scheduling and cause stalls. Seen on very high bitrates.
• added cipher list tag for compatibility with recent icecast 2.4.x builds
• pass through unknown metadata on relayed ICY through to the requesting listeners.
• some minor tunings to help large bitrate cases.
• A few tweaks for restarting on-demand relays on failure cases.
• A few bad pointer cases fixed, typically restarting relay stats, hard to trigger.
• added UTF8 check on updated stats routines.
• added content-disposition header for xsl sending non-html content.
• minor changes to YP scheduling/logging.

18/02/2014

• xslt update race fix
• Format. mpeg parsing can be bypassed if mpeg4 is detected. mpeg assumed by default even application/octet for streams only. EBML streaming working again. Ogg handler was getting stuck with large pages, typically intro via libcurl.
• Auth. Fix up intro content from auth. Presume innocent setting was being ignore if the queue was full. Added referer to auth command
• Relay updates. uses single structure for all relays, with timestamps. makes better handling of startup, shutdown and changes to relays. A redirect response handled better, if user/pass returned. Queue cleanup on inactive relay, reduce memory use in such cases.
• issue CORS headers, just wildcard for now.
• yp race fixed, make sure yp_currently_playing is not dependent on YP add completing
• stats client bug on initial stats dump fixed
• various internal setting tweaks to trigger events or limit actions.
• various lockup and crash cases fixed.

16/08/2013

• Allow for ${mount} to be specified in dump-file, on-[dis]connect, fallback-mount and intro. Useful in cases where a wildcard mount block is used but requires these options
• avoid possible corruption case with FLV listeners and a change in samplerate
• move trigger for override later. When first block added to queue.
• delay purging of file handles as they could be referenced by inactive mounts for slaves
• fix for outgoing bitrate and global listeners stats not updating in some cases.
• tunings and code re-arrangement to prevent busy loops.
• fix for passed through content.
• prevent possible live lock on sources with limit rate set.
• fix for http parameter parsing
• stats send link improved. Not many use this but could be used by slaves later on
• build fixes

23/05/2013

• processing certain mpeg intro file could lead to a listener disconnection.
• Auth remove could be triggered twice if no stream content comes in before time out
• Incorrect content type sent for aac files
• The lag check on the queue could trigger a crash, One case was handled now both are.
• Drop SO_REUSEADDR for win32, can break things over reload.
• Make file requests use normal HTTP response, while streams go through the agent checks. It seems that for asx playlists, wmp didn't handle ICY, but requires it for streams
• Fix up the log size setting when using the 2.3 way or doing it. it was being ignored
• Add ID back into per-listener details as some use 2.3.2 pages, typically centova users
• 64bit type clean up.

24/04/2013

• processing certain mpeg intro file could lead to a listener disconnection.
• initialise mpeg resync count on first frame found, prevents looping fallback files with ID3 or other content from dropping listener after so many times.
• fix relays not overriding listeners from a fallback when starting up.
• prevent the (n appearing in metadata for mp3/aac streams.
• allow for source queue shrinkage. With the rwlock merging, the queues were left to expand to queue-size whether listeners were lagging or not, this helps reduce unwanted parts of the queue.
• Avoid listener remove event if source client passes auth but is rejected due to the source being in use
• introduce lazy slave stats update for updating virtual mountpoints, this reduces the stats generated if you have many mountpoints with frequent starts and stops.
• drop PE substitution in metadata, only do the likes of &
• allow for listener processing to be skipped if it would stall waiting on lock.
• stats link update, like issuiing NEW commands first at connection before EVENT, not important for most people
• Various log message updates and small scale lock range updates.

18/02/2013

• update response header code for returning 200/206. On streams, do not send 206, but allow it for file requests.
• migrate some legacy source stats calls to newer API, reduces workload.
• fix some corner cases for queue handling.
• tweaks for response headers for iphone and blackberry
• fix for some cases where listener_remove is not triggered.
• fix for override fallback file through cascade.
• do not use spin locks on BSD for now, use mutexes.
• mpeg parsing for processing files handled better now.
• fix for handling privileged ports over reload.
• fix for SOCK_CLOEXEC on linux where it is not implemented
• allow for multiple args to be supplied to on-[dis]connect scripts.
• various performance tunings all over, along with a few race fixes and memory leaks

12/09/2012

• crash fix with intro files. intro reader requires larger buffer

2.3.3-kh4, Use kh5 instead

09/09/2012

• burst size of less than 1400 was not handled correctly with recent rwlock changes
• some memory reduction for FLV listeners.
• Missing xsl could cause a busy loop.
• fixed a crash when listener reached end of the queue. Certain assumptions about buffer of queue accounting were changed with rwlocks, exposed a crash case.
• on some multi-processor setups, it was possible that certain listeners could trigger a busy worker due to old (cached?) data, work around applied.
• A listener that started off a new on-demand relay could be only partially initialised a lead to a crash later on
• report of recent flash update on FF15 when requesting non-FLV content. Expected header identifying flash is no longer present so now we check for swf in referer.
• Add short write handler for clients requesting short ranges (eg Apple)
• The stats handle for normal file or fallback files were altered in a previous release which prevented a lookup of listener details via admin requests. now fixed.
• only move inactive relay client to another worker when there is sufficient difference
• honor auth options "allow_duplicate_users" and "drop_existing_listener" settings for webroot files
• proxy cache avoidance headers more generally used.
• minor memory leak on moving clients over fallback.
• code cleanup

07/08/2012

• crash fix with invalid stats handle with fallback to file
• merge Opus module from trunk
• Force NTPL to prefer writers over readers for rwlocks
• bitrate estimation scales better now.
• where possible, use pread to avoid mutex locks on file reads, win32 needs fixing up
• make xsl lookup cache scale better, force a reload via a separate thread to prevent worker stalls.
• various tunings for client rescheduling
• small changes to client balancing across workers
• small changes to log messages.
• reference a git hash code is available

15/07/2012

• sent value passed to listener_remove in kh1 could be wrong on 32bit setups
• avoid clearing queue on relay restart, it exposed a crash case with lock changes
• make TCP sockets not pass to on-[dis]connect scripts. useless work avoided
• hijack source fix required with lock changes, to avoid mis-calculated lags drops
• refcount bug missed for sending to master relays. possible memory corruption
• more worker balancing changes, to spread loading more evenly.
• missing delay for blocking file client, has an effect of higher cpu usage
• rare override failure case handled better.
• various minor fixes, including log format, memory leak and stats.

24/06/2012

• Big change with rwlocks replacing mutexes. It's internal but can have a big effect on load pattern usage. In cases with many listeners on a single mount we can now spread clients across all workers. A number of balancing changes were done to assist in localising related clients but may need more tweaking. A side effect of these changes is that queues are only shrunk when there are no listeners, so queues can take up more memory than usual but that could of happened before.
• fix bad reference for file-based content like intro/fallback with FLV
• make the minimum duration of an on-demand relay last until it would be classed as working, any less then it could be skipped or disabled for a short time.
• skip entity decoding if none required. For cases where metadata has & causing the libxml2 to spit out warnings like "no name".
• If no sockets are opened at the start then exit.
• Win32. standardize on format for date, back to proper timezone numbers
• Log AAC samplerate change in stream.
• Add <inactivity-timeout> in <limits> to stop icecast when there are no sources
• Add <x-forwarded-for> into <paths> for IP substitution from load balancers
• Added referer= to listener_add POST.
• Added sent= to listener_remove POST (bytes)
• sync ebml/webm format parser with 2.3.3, not tried it myself.
• minor tweaks for rescheduling on high bitrate streams, terminating sources, URL auth handling if disabled and busy relay restarting.

19/05/2012

• Expand file open limit to match <clients> if possible, warn if less.
• Fix possible client leak. legacy code removal.
• updated mpeg parser to be more specific on bits to match for. It should increase the tolerance for changes in the stream format (CRC, samplerate etc).
• reload forces an update of active streams, was missing from a previous update.
• Realplayer based apps fix for aac+. Assumes certain response header.
• drop requirement for matching each / in mount-name.
• content type needed truncating in rare cases before format could be set up
• prevent too much being read in one go when reading ogg data.
• internal change to swapping of hijacked clients, possible crash fix.
• fallback over multiple mounts works again
• Add timeout for sync mode handling. Should never trigger really.
• a few small log reporting changes.

17/04/2012

• Reduce memory copies with large stream lists from master
• internal FLV buffer management cleaned up, possible crash case avoided.
• return 501 instead of 400 for HEAD requests, helps some players.
• stats updating reduced now that relays always have clients, also reduce locking.
• missing user/pass setting for metadata update from shoutcast style source and url auth
• Experimental. send variable block framing if requested by listener
• Experimental. allow <include>/dir/*.xml</include> in <icecast> for breaking xml up
• Experimental. webm work by David Richards
• Experimental. source client hijacking. replace existing source with another. has to to allowed via auth url currently to prevent bouncing of clients with auto-reconnect.
• Add duration to all access, error and playlist logs, allows re-opening them after a number of seconds elapsed, 0 by default. set size trigger to 50Meg default.
• possible busy loop in kh31 in cases where fallback is defined
• fix for win32 error reporting and missing uninstall link in programs list
• added "timeout" (default 5) and "on_error_wait" (default 60) to auth url options.
• sync up headers handled by auth cmd with auth url.
• Added missing stream format checks for certain error cases.
• speedup for icy metadata and avoid length bug on inline urls.
• A number of possible lock fixes and checks are now added for odd cases.