Game Sounds Using HTML5 and RX-JS

With HTML5 and Javascript, it's pretty easy to play sounds in the browser. And when you're using Reactive Extensions for Javascript, it's also pretty easy to "bind" sound effects to events. I created a simple "sound engine" to my Worzone game experiment and used RX to bind sounds to events. The source code is of course available in GitHub at https://github.com/raimohanska/worzone and you can have a quick demo at http://juhajasatu.com/worzone/. Just click on the "sound" checkbox before pressing any key to hear the stuff you're looking for.


Anyway, I'll briefly show you the code here too. Let's start with the "sound engine", which looks like this:


function Audio() {   
  var on = false

  var sounds = {}


  function loadSound(soundName) {
    var audioElement = document.createElement('audio')
    audioElement
      .setAttribute('src', "audio/" + soundName + ".ogg")
    return audioElement
  }


  function getSound(soundName) {
    if (!sounds[soundName]) {
      sounds[soundName] = loadSound(soundName)
    }                                         
    return sounds[soundName]          
  }              
  function play(soundName) {   
    if (on) getSound(soundName).play()
  }
  return {
    // String -> (Unit -> Unit)
    playSound : function(soundName) { 
                  return function() { play(soundName) }
                },
    // Unit -> Unit
    toggle : function() { on = !on; }
  } 
}


The Audio object exposes two methods, one of which is used to toggle sound on/off and the other returning a function that will play a given sound when invoked. The comments above the methods use the Haskell syntax for describing what are the input and output types of the functions. 





One funny thing about HTML5 audio is that .ogg format is supported while .mp3 is not. Some patent issues there, I guess. The reason why playSound returns a function instead of playing a sound is that this way it's easier to plug it into RX streams, as in the following piece of code which constitutes the whole of game sound related code in Worzone:



function GameSounds(messageQueue, audio) {
  function sequence(delay, count) {
    return ticker(delay)
      .Scan(1, function(counter) {return counter % count + 1} )    
  }
  sequence(500, 3)
    .SkipUntil(messageQueue.ofType("level-started"))
    .TakeUntil(messageQueue.ofType("level-finished"))
    .Repeat()
    .Subscribe( function(counter) { 
       audio.playSound("move" + counter)() 
    })
  messageQueue.ofType("start")
    .Where(function (start) { return start.object.player })
    .Select(function(start) { return start.object.player.id })
    .Subscribe(function(id) { audio.playSound("join" + id)() })    
  messageQueue.ofType("fire")
    .Subscribe(audio.playSound("fire"))
  messageQueue.ofType("hit")
    .Subscribe(audio.playSound("explosion"))
  messageQueue.ofType("level-starting")
    .Subscribe(audio.playSound("intro1"))
} 


The messageQueue thing there is an RX Observable that I described in one of my previous postings. It will provide all game events, each containing a message property that can be used for filtering events. The ofType method actually provides an Observable containing only messages of the given type.


The actual sound effects are

• An alternating sequence of move[1..3] sounds played each 500 milliseconds, starting when a game level is started and ending when a level ends. So, no creepy beeping when the monsters aren't moving.
• Two sounds join[1..2] that are when player 1 or 2 starts the game.
• Fire, explosion and intro sounds that are played when somebody fires or dies and when a new game level is about to start.

There seems to be some problem with my formatting. Extra line breaks appear. Not in the editor though. What's up?

Failing with RX-JS

The first-ever example I've seen of Reactive Extensions for Javascript is the example, where you've got these +/- buttons that you can use to increment/decrement a counter:

The code is nice and elegant as in



$(function() {
function always(x) { return function(_) { return x }}
var incr = $('#incr').toObservable('click').Select(always(1))
var decr = $('#decr').toObservable('click').Select(always(-1))
var series = incr.Merge(decr)
.Scan(0, function(total, x) { return total + x })
series.Subscribe(function (total) { $('#count').html(total) })
})

Great, isn't it? Well, I wasn't quite impressed. This is a toy example and doesn't really show the real power of RX. However, what do you think will happen with the following addition:



  $('#moar').click(function(){ 
    series.Subscribe( function(total) {$('#count2').html(total)})
  })

I'd expect that after clicking on the "moar" button, a duplicate counter would appear, showing the same number as the first one. However, it'll look like this:

How come? It seems that when a new Subscriber is added, it will get a new copy of the Observable series, starting from zero. It seems that the accumulator in the Scan combinator is not shared by subscribers. I guess it's logical but it certainly took my by surprise. It doesn't feel right that different observers of the same Observable see different values. What do you think? 

This can be fixed quite easily though:

  var series = incr.Merge(decr)

    .Scan(0, function(total, x) { return total + x })

    .Publish()

  series.Connect()

Not hard per se, but the real problem with RX is that you have to know all these tricks and there's no-one there to teach you (except some brave bloggers). Where is the public API documentation? Well, it's there for sure. Provided in Windows Help File format, in the file RxJS.chm that's bundled with the distro. Who uses .chm files for documentation?

According to Microsoft's fine documentation, the Publish() method "Returns a connectable observable sequence that shares a single subscription to the underlying source". It doesn't say that you have to call Connect() too to make it actually work. But hey, it doesn't say much else either. 

I HATE the way this library is documented. It makes me feel the authors are either cocky or stupid. The latter seems impossible regarding the quality of the actual code.

EDIT: Sorry for the last line. I've got huge respect for the Rx team and by now they've improved documentation a lot too, but that was the situation and those were my feelings at the time I wrote this post.

Creating Custom Observables and Combinators in RX JS

I'm writing this because I feel there's not much stuff available on writing custom Observables in Reactive Extensions for Javascript.


1. The Basics


First, I had to grasp how the subscribe/unsubscribe mechanism works. And it goes like this:


1. You've got an observable


  var observable


2. You subscribe, and you get a handle for unsubscribing


  var disposable = observable.Subscribe(function(message){...})


3. You unsubscribe using the Dispose() method


  disposable.Dispose()

If you're not working on custom Observables or Combinators, you'll hardly ever need the Dispose() function, as you'll more likely use methods like Take, TakeUntil to select how many calls you really want your Subscriber to get.


When creating my custom Observables, I have to make sure I return a working "dispose" function from the Subscribe function. For example, for a simple "ticker", it could work like this:


function ticker(interval) {

  return Rx.Observable.Create(function(observer) { 
   var id = setInterval(observer.OnNext, interval) 
   return function() { clearInterval(id) }
  })
}   

So for a custom Observable, you need to provide a Subscribe function that's called whenever a new Subscriber is added. This function returns another function, that's used by RX when Subscriber is removed.

2. The SampledBy combinator

In my Worzone project, I had the need to sample an Observable each 50 milliseconds. Observable.Sample() doesn't seem to work for me, because it only takes sample if there's a new event in the source stream. In my case, the source stream  (a stream containing the current direction where a figure should go) gets events only when something changes. I still want to get a sample (the latest value) every 50 ms. Hence, a custom combinator!

First, I created a more general-purpose combinator called CombineWithLatestOf, that combines a stream with the latest value of another stream, using a given combinator function:

Rx.Observable.prototype.CombineWithLatestOf = function(otherStream, combinator) {    
  var mainStream = this
  return Rx.Observable.Create(function(subscriber) {        
    var latest
    var d1 = mainStream.Subscribe(function(mainValue) { 
      subscriber.OnNext(combinator(mainValue, latest)) 
    })
    var d2 = otherStream.Subscribe(function(message) { 
      latest = message
    })
    return function() {
      d1.Dispose()
      d2.Dispose()
    }
  })
}    


Here you see the Dispose() method in action. When a Subscriber is added to the result stream, new Subscribers are added to both the source stream and the other stream (where we want the latest value from). When the Subscriber is removed, the Dispose() method is called for both "proxy" subscriptions. I challenge you to tell me what's missing from above, or how the same could be done in a more elegant fashion!


Anyways, this combinator makes it easy to implement SampledBy:



Rx.Observable.prototype.SampledBy = function(otherStream) {
  return otherStream.CombineWithLatestOf(this, function(a,b) {
    return b
  })
}

3. The MessageQueue Observable

The other major thing I couldn't find in RX-JS is a "message queue", where I can push messages and that implements Observable, delivering all the pushed messages to Subscribers. To go a bit deeper, I implemented a plug method that allows me to plug in other streams to the MessageQueue. The result is that I can plug all streams into this central bus that I can then filter to observe certain types of messages.

And here it is.

function MessageQueue() {     
    function remove(xs, x) {
       xs.splice(xs.indexOf(x), 1)
    }      
    function Subscription(observable) {
      var disposable              
      function cancel() { remove(subscriptions, subscription)}                                   
      function push(message) { messageQueue.push(message) }
      function start() {
        disposable = observable.Subscribe( push, cancel)        
      } 
      function stop() {
        if (disposable) disposable.Dispose()  
      }                   
      var subscription = {
        start : start, stop : stop
      }                 
      subscriptions.push(subscription)
      if (observers.length > 0) { start() }
      return subscription;
    }                                 
    var subscriptions = []
    var observers = []    
    var messageQueue = Rx.Observable.Create(function(observer) {                               
        observers.push(observer)
        if (observers.length == 1) {
          subscriptions.forEach(function(subscription) { 
            subscription.start() 
          })
        }
        return function() { 
          remove(observers, observer); 
          if (observers.length == 0) {
            subscriptions.forEach(function(subscription) { 
              subscription.stop() 
            })
          }
        }
    })    
    messageQueue.push = function (message) {         
        observers.map(identity).forEach(function(observer) {
            observer.OnNext(message)
        });
        return messageQueue
    }
    messageQueue.plug = function (observable) {
        Subscription(observable)
        return messageQueue
    }    
    return messageQueue
}        


Whoa. Sorry to spit out such a large chunk of code here. At first it was very simple, but then I discovered that I had been cheating in bookkeeping, I mean, not cleaning up all observers properly. That's quite a lot of bookkeeping there. I wish I could cut it down somehow. Can you?

Game Programming With RX-JS

I'm quite excited by Microsoft's RX (Reactive Extensions) for JavaScript. I 'm also nostalgic for old school Commodore 64 games such as Bruce Lee, Archon and especially Wizard of Wor. Would'nt it be be cool to be able to play WoW(!) online.. So, why not give it a try and start writing it in RX-JS. 


First, I want to be able to control the Pixel Man (his real name remains unknown to me):



1. Keyboard Control


So, I started by defining some streams. This is actually the hardest part, so bear with me for a while.

function keyState(keyCode, value) {
  return Rx.Observable.FromArray([[]])
    .Merge(keyDowns(keyCode).Select(always([value]))
      .Merge(keyUps(keyCode).Select(always([])))
        .DistinctUntilChanged())
}

var allKeyUps = $(document).toObservable("keyup")
var allKeyDowns = $(document).toObservable("keydown")

function always(value) { 

  return function(_) { return value } }

function keyCodeIs(keyCode) { 

  return function(event) { return event.keyCode == keyCode} }

function keyUps(keyCode) { 

  return allKeyUps.Where(keyCodeIs(keyCode)) }

function keyDowns(keyCode) { 

  return allKeyDowns.Where(keyCodeIs(keyCode)) }

.. and I've got a stream of a given key's state, mapped into a single-element array in case the key is down, or the empty array if the key is up. Like below..

allKeyDowns           ...keydown......................

                         |

allKeyUps             ...|.............keyup..........

                         |             |              

keyState(key, "LOL")  ...["LOL"].......[].............

The keyState function starts with the empty array, and merges in all keyDowns as single-element arrays and all keyUps as empty arrays.

Then, I combine a bunch of these streams so that I can get the state of multiple keys are a set of values:

function multiKeyState(keyMap) {
  var streams = keyMap.map(function(pair) { 
    return keyState(pair[0], pair[1]) })
  return Rx.Observable.CombineLatestAsArray(streams)
}

I created some custom RX combinators for this:

Rx.Observable.CombineLatestAsArray = function(streams) {   
  return Rx.Observable.CombineAll(streams, function(s1, s2) { 
    return s1.CombineLatest(s2, concatArrays)})  
}

Rx.Observable.CombineAll = function(streams, combinator) {

  var stream = streams[0]

  for (var i = 1; i < streams.length; i++) {

    stream = combinator(stream, streams[i])

  }

  return stream;

}

function toArray(x) { return !x ? [] : (_.isArray(x) ? x : [x])}

function concatArrays(a1, a2) { 

  return toArray(a1).concat(toArray(a2)) }

This finally allows me to map the arrow keys into direction vectors that define where the Pixel Man shall go:

var keyMap = [

  [38, Point(0, -1)],

  [40, Point(0, 1)],

  [37, Point(-1, 0)],

  [39, Point(1, 0)]

]

var direction = 

  multiKeyState(keyMap).Where(atMostOne).Select(first)

The direction stream will then map my keystrokes like below:

  keydown:left......keyup:left....keydown:right

  |                 |             |

  Point(-1, 0)      undefined     Point(1, 0)

.. the Point function naturally returning simple x/y pairs.

2. Movement

To make the man move, I will start with a startPos, sample the direction stream every, say 100 ms, and increment the position by the currect direction. Simple as that:

function identity(x) { return x }

var ticker = Rx.Observable.Create(function(observer) { setInterval(observer.OnNext, 100) })

var movements = ticker.CombineLatest(direction, function(_, dir) { 

  return dir }).Where(identity)

var position = movements.Scan(startPos, function(pos, move) { 

  return pos.add(move.times(4)) })

I started by constructing a ticker stream that will generate an event every 100 ms. The movements stream was created by mapping the ticker events into the current direction using CombineLatest, and filtered out the undefined values by using Where(identity). Finally, the position stream is a kind of a sum of all movements, with the starting value of startPos.

At this point I might mention that the Point class that represents position and movement vectors (x/y pairs) has some methods for adding and multiplication, as can be seen in the defining function of the position stream. Please have a look at the full source code.

3. Graphics

So far this has been functional programming, without any side effects or mutable state. As you can see, I'm tracking keyboard state and Pixel Man position without any explicit state variables, thanks to RX!

To make the Pixel Man materialize and start moving, we need to select some graphics framework and assign side-effects to the position stream. So, I'll start by initializing Raphael and adding the Pixel Man on a black background:

var bounds = Rectangle(0, 0, 640, 480)

var r = Raphael(10, 10, bounds.width, bounds.height);

r.rect(bounds.x, bounds.y, bounds.width, bounds.height)

  .attr({fill : "#000"})

var startPos = Point(100, 100)

var man = r.image("man1.png", startPos.x, startPos.y, 40, 40)

Now that I've got the man on the stage and I've got the streams set up, I can make him move with a nice one-liner:

position.Subscribe(function (pos) { 

  man.attr({x : pos.x, y : pos.y}) })

4. Animation and Rotation

It was a bit dull to see the Pixel Man float around, even though he was in my control, so I added animation:

var animation = movements.Scan(1, function(prev, _) { 

  return prev % 2 + 1})

animation.Subscribe(function (index) { 

  man.attr({src : "man" + index + ".png"})}) 

Now that was cool, wasn't it? No poking around the code, just one new stream and a side effect. The stream maps movements in to an alternating series of 1's and 2's. The side-effect alters the image between the two png's that I've got.

Finally, I made the man look where he's going, instead of just looking right:

var angle = direction.Where(identity).Select(function(vec) { 

  return vec.getAngle()})

angle.Subscribe(function(angle) { 

  man.rotate(angle * 360 / (2 * Math.PI) + 180, true) })

Same thing here: a stream of angles and a side effect that rotates the Pixel Man.

5. Some fixing

I was happy with the simplicity and elegance of how I implemented animation and rotation. However, I wasn't so happy with the man turning upside-down when he was moving to the right.. So, I replaced the elegant code with something a little less elegant, but still quite simple:

var animAndDir = direction.Where(identity)
  .CombineLatest(animation, function(dir, anim) { 
    return {anim : anim, dir : dir}})
animAndDir.Subscribe(function(state) {
  var angle, basename
  if (state.dir == left) {
    basename = "man-left-"
    angle = 0
  } else {
    basename = "man-right-"
    angle = state.dir.getAngle() * 360 / (2 * Math.PI)
  }
  man.rotate(angle, true)
  man.attr({src : basename + (state.anim) + ".png"})
})

6. Conclusion

Easy wasn't it? After some learning and having mastered the keyboard state, it was quite trivial to make the Pixel Man move, rotate and animate.

Now I have a black screen with two keyboard-controlled Pixel Men. I've played it with my daughter and she coined it the Robot Game.  At 1 year and 11 months it's freaking awesome to be able to control a robot.

<1 week later>

After I wrote this, I hacked some more features into the game. Now there are two players and also some enemies there.

The code is available at https://github.com/raimohanska/worzone
Plx check out a live demo at http://juhajasatu.com/worzone/

Worx great in Chrome and Safarei, but a bit slowly in Firefox. Just like with the Lavalamp I blogged about earlierly.