Network Protocol

From TomahaWiki

Jump to: navigation, search

Tomahawks connect to each other over the network to share music databases and stream music. This page describes the network protocol.

Contents

Local Network Discovery

Tomahawk advertises its presence locally by broadcasting a single packet on port 50210. The packet contains only a string like this TOMAHAWKADVERT:50210:66bd135d-113f-481a-977e-dea80cz7b338c where 50210 is the TCP port that Tomahawk is listening on and 66bd135d-113f-481a-977e-dea80cz7b338c is that Tomahawk's UUID.

This advertisment packet is sent once at startup and then again every 60 seconds.

Each Tomahawk is also listening for the advertisement packets. Incoming advertisements are only processed if they are sent by an IPv4 link-local IP, a lookback IP, or one of the private network IPs. If the received advertisement contains a UUID that Tomahawk is not already connected to, then Tomahawk will create a ControlConnection. Details about what happens next are in the next section.

Connections

Every Tomahawk listens and accepts connections on a single TCP port. Tomahawks use three types of TCP connections to communicate with eachother: ControlConnection, DBSyncConnection, and StreamConnection. Two Tomahawks are considered connected if they have an active ControlConnection. Once connected either side can open a DBSyncConnection to sync their music databases or a StreamConnection to stream audio.

Message Framing

All messages sent over the TCP connections share the same message framing. Each message starts with a 5 byte header. The first 4 bytes are a big-endian integer which is payload length in bytes. The following byte is a flags bitfield. The rest of the message is the payload.

Header (5 bytes) Payload
Payload length (4 bytes) Flags (1 byte) Message Payload (0-N bytes)

Flags

The flags bitfield is ORed together from these 

       RAW        = 1
       JSON       = 2
       FRAGMENT   = 4
       COMPRESSED = 8
       DBOP       = 16
       PING       = 32
       RESERVED_1 = 64
       SETUP      = 128

JSON: the msg payload is JSON representing a javascript object.

COMPRESSED: the payload has been compressed using QT's qCompress function which I think uses zlib. To uncompress using python's zlib module you first need to remove the initial 4 byte header which contains a big-endian integer that is the uncompressed size of the payload in bytes. The compressed flag is only used and supported in ControlConnections and DBSyncConnections.

FRAGMENT: indicates that a command will be followed by more. When sending db operations, all but the last DBOP message will be flagged with FRAGMENT. Similarly when streaming a song all but the last message will have the FRAGMENT flag set.

RAW: Set on messages sent in StreamConnections after the initial message and SETUP messages.

DBOP: a DB OPeration sent in DBSyncConnections.

PING: used in ControlConnections on empty ping messages that are periodically sent by both sides.

SETUP: set in the version check and version response messages.

Initiate Connection

When a Tomahawk first connects it will send a message list this: 

{
   "conntype": "accept-offer",
   "nodeid": "66bd135d-113f-481a-977e-111111111111", 
   "key": KEY, 
   "port": 50210
}

The conntype "accept-offer" is sent when the connection is made to a host found through some other means like SIP or a received advertise packet. The nodeid is the UUID of the Tomahawk sender.

Secondary DBSyncConnections and StreamConnections start with a msg like 

{
   "conntype": "accept-offer",
   "controlid": "66bd135d-113f-481a-977e-111111111111", 
   "key": KEY, 
   "port": 50210
}

The presence of the "controlid" field instead of the "nodeid" field tells you that this is a secondary connection. There should be an existing ControlConnection that started with the same UUID as its "nodeid".

Setup and Version Check

The first message that the Tomahawk that accepts a connection sends is a setup message. The msg has the flag SETUP and the payload "4", where 4 is the protocol version. The other Tomahawk replies with a flag SETUP and payload "ok", if the version matches. If not, the reply with be the JSON 

{"method":"protovercheckfail"}

If the check fails, the connection is shutdown. If the version check succeeds, Tomahawk starts using the connection.

If this setup exchange is not completed within 3 minutes, then the connection is closed.

Connection Keys

The "key" field in the initial connection message determines how the connection will be used.

Key: "whitelist"

Starts a ControlConnections between Tomahawks on a local network.

Key: "FILE_REQUEST_KEY:..."

Starts a StreamConnection.

Key: UUID

Used when a Tomahawk creates a new DBSyncConnection or StreamConnection in response to a request from the other Tomahawk.

Control Connections

Two Tomahawks are are connected when they an active ControlConnection.

Requesting Connections

If two Tomahawks are communicating over the internet, only one of them may have an open port, so only one of them may be able to connect. If a connected Tomahawk wants to open a Stream or DBSync connection but is unable to it can ask the other Tomahawk to do so by sending a message like this 

{
   "conntype": "request-offer",
   "controlid": "66bd135d-113f-481a-977e-111111111111", 
   "key": KEY,
   "offer": OFFER_KEY,
   "port": 50210
}

Which causes the other Tomhawk to create a new connection back using a conntype "push-offer" and KEY as the key. The OFFER_KEY determines the type of connection. The first message sent on the new connection is 

{
   "conntype": "push-offer",
   "controlid": "66bd135d-113f-481a-977e-222222222222", 
   "key": KEY,
   "port": 50210
}

Pings

Every 5 seconds Tomahawk will send an empty message with the flag PING over the ControlConnection. If a Tomahawk doesn't receive the PING message for 10 minutes it will disconnect.

The ping messages aren't a request and there isn't a corresponding pong message. Both sides in the connection send ping messages on a timer.

Database Synching

To sync databases, one Tomahawk sends this msg on the ControlConnection 

{
   "method": "dbsync-offer",
   "key": "66bd135d-113f-481a-977e-111111111111"
}

Then the other Tomahawk will either open a DBSyncConnection or request that the original Tomahawk open a connection.

DB Operations

Each Tomahawk maintains a database containing its local music collection. Each change to this database is serialized as a database operations with an assigned UUID. These operations can be sent to other Tomahawks so that they can construct a copy of the music collection. By fetching database operations that have occurred after a certain UUID, a Tomahawk can efficiently keep it's copies up to date by applying only the operations it has not yet seen.

Fetching DB Operations

To request database operations, Tomahawk sends the following message 

{
   "method": "fetchops",
   "lastop": "" 
}

an empty "lastop" string asks for all operations. If "lastop" is a UUID then only operations that occurred after the one identified will be sent. However, the "guid" of some commands cannot be used in "lastop". Currently, the ones that can't be used are "logplayback" commands with "action" = 1. That may change in the future, but hopefully a new field will be added to all of the db commands to indicate which cannot be used.

In response, the other Tomahawk will start sending messages flagged with DBOP. If there are no new operations, then a single DBOP msg will be sent containing "ok". If there are one or more operations, each will be sent in a separate message and all but the last message will be flagged with FRAGMENT. Each database operation message looks like 

{
   "command": COMMAND_TYPE,
   "guid: GUID,
   ... additional fields ...
}

where COMMAND_TYPE and GUID are strings.

Database Operation Types

AddFiles

Contains a list of added files. 

{
   "command": "addfiles",
   "guid": "7ea0b94a-498f-4c1a-a9ac-afb6b11d3a3d",
   "files": [
     {
       "id": 1, 
       "url": "1", 
       "artist": "Bloc Party", 
       "album": "Banquet", 
       "track": "Banquet", 
       "mimetype": "audio/mpeg", 
       "hash": "", 
       "year": 2002, 
       "albumpos": 1, 
       "mtime": 1261233412, 
       "duration": 201, 
       "bitrate": 128, 
       "size": 3229824
     },
     ...
   ]
}

Deletefiles

Contains a list of file IDs that have been deleted. 

{
   "command": "deletefiles", 
   "guid": "f32a1bed-9774-48ec-b90f-71795fab94d8", 
   "ids": [351, 352, 353, 354, ...]
}

CreatePlaylist 

{
   "command": "createplaylist", 
   "guid": "235517ef-488d-4d35-a5c3-6e2092aa0edd",
   "playlist": {
       "info": "", 
       "creator": "My Collection", 
       "createdon": 1321308437, 
       "title": "New Playlist", 
       "currentrevision": "", 
       "shared": false, 
       "guid": "d333b5f7-fda3-4f58-a387-18f47ca02b6d"
   }
}

RenamePlaylist 

{
   "command": "renameplaylist", 
   "guid": "60f995a0-305e-49e4-a99c-b445f51a71af", 
   "playlistguid": "d333b5f7-fda3-4f58-a387-18f47ca02b6d", 
   "playlistTitle": "Greatest"
}

SetPlaylistRevision

Adding songs to the playlist causes a setplaylistrevision command. 

{
   "command": "setplaylistrevision", 
   "guid": "4181526a-1298-4130-8b1d-59eea3a8249a",
   "oldrev": "", 
   "newrev": "012f4a07-0362-4db3-a886-4c7766a17889", 
   "playlistguid": "d333b5f7-fda3-4f58-a387-18f47ca02b6d", 
   "addedentries": [
       {
           "duration": 4294967295, 
           "lastmodified": 0, 
           "guid": "77b113ae-6ee0-4e63-bcd2-94cb3dc3990c", 
           "annotation": "", 
           "query": {
               "album": "Yoshimi Battles the Pink Robots", 
               "duration": -1, 
               "qid": "b8e24961-7041-48db-9f41-161f4aadf362", 
               "track": "Yoshimi Battles the Pink Robots, Part 1", 
               "artist": "The Flaming Lips"
           }
       }, 
       {
           "duration": 4294967295, 
           "lastmodified": 0, 
           "guid": "12ef55df-cbe1-4756-be71-b2209bcd6199", 
           "annotation": "", 
           "query": {
               "album": "Yoshimi Battles the Pink Robots", 
               "duration": -1, 
               "qid": "0cf3bd28-35aa-4711-ab4e-182a92b55cfd", 
               "track": "Yoshimi Battles the Pink Robots, Part 2", 
               "artist": "The Flaming Lips"
           }
       }
   ], 
   "orderedguids": [
       "77b113ae-6ee0-4e63-bcd2-94cb3dc3990c", 
       "12ef55df-cbe1-4756-be71-b2209bcd6199"
   ]
}

Only newly added songs are included in the "addedentries" list, but every each setplaylistrevision contains an "orderedguids" list with the entire playlist. Removing a song creates a setplaylistrevision with an empty "addedentries" list and the new "orderedguids" with only the remaining songs. 

{
   "command": "setplaylistrevision", 
   "guid": "966a99ac-d8ca-4e64-88de-8625b5b529bc"
   "oldrev": "012f4a07-0362-4db3-a886-4c7766a17889", 
   "newrev": "32aaed70-c262-439e-a3f6-b50197903815", 
   "playlistguid": "d333b5f7-fda3-4f58-a387-18f47ca02b6d", 
   "addedentries": [], 
   "orderedguids": [
       "77b113ae-6ee0-4e63-bcd2-94cb3dc3990c"
   ]
}

DeletePlaylist 

{
   "command": "deleteplaylist",
   "guid": "54e6e9d4-aeb1-4ecc-9031-eff060ec0540", 
   "playlistguid": "d333b5f7-fda3-4f58-a387-18f47ca02b6d"
}

SocialAction

Sent when you love or unlove a song.

Loving a song: 

{
   "command": "socialaction", 
   "guid": "8675227b-f8f8-4cfa-9e38-b155383f1460",
   "action": "Love", 
   "comment": "true", 
   "artist": "Nirvana", 
   "track": "The Man Who Sold The World", 
   "timestamp": 1321460771    
}

unloving a song: 

{ 
   "command": "socialaction", 
   "guid": "316d02cc-ec4b-49fa-9ecf-6031f451faa9"
   "action": "Love",
   "comment": "false", 
   "artist": "Nirvana", 
   "track": "The Man Who Sold The World", 
   "timestamp": 1321460775
}

SetCollectionAttributes 

{
   "command": "setcollectionattributes", 
   "guid": "1e3b904c-4120-474b-9151-4c2c7f72de6e", 
   "del": false, 
   "id": "CAURJBB133ACEB252B", 
   "type": 0
}

"id" here is the echonest song catalog id that contains all the songs in the collection.

SetTrackAttributes

LogPlayback

Notifies when Tomahawk starts or stops playing a song. When a song starts playing this is sent 

{
   "command": "logplayback", 
   "secsPlayed": 0, 
   "artist": "Beastie Boys", 
   "track": "Brass Monkey", 
   "action": 1, 
   "playtime": 1321454618, 
   "guid": "a2fd565f-99a5-45c2-bb38-a881cfa20c20"
}

When a song stops playing, this is sent 

{
   "command": "logplayback", 
   "secsPlayed": 56, 
   "artist": "Beastie Boys", 
   "track": "Brass Monkey", 
   "action": 2, 
   "playtime": 1321454681, 
   "guid": "acb19088-0b58-425b-a0bd-df648014ad04"
}

The "action" number can be one of two values: 1 is started, 2 is finished. Be careful never to fetchops using the guid of a logplayback which has action 1. Tomahawk does not store these in its database so they can't be used to query for dbops. In fact, you'll crash Tomahawk if you try to.

Triggering Fetches

When Tomahawks first connect they will automatically send a "fetchops" message to get the latest changes. When a Tomahawk makes changes to its local music collection it will send trigger messages to all connected Tomahawks to notify them of the changes. In response, the other Tomahawks can then send "fetchops" messages to get the new changes. The trigger message looks like this 

{"method": "trigger"}

Music Streaming

On request, Tomahawk will stream the contents of any file in its local collection to another connected Tomahawk. The files are streamed as separate messages in a StreamConnection and seeking is supported.

To start streaming, create a connection with a key such as "FILE_REQUEST_KEY:42" where "42" is the id of a file from an AddFiles DB operation. After the connection message and the version check messages, Tomahawk will begin sending messages with flag RAW. The payload of each message starts with the 4 bytes "data". The rest of the payload is a block of binary data, straight from the file. The amount of binary data in each message is hardcoded in Tomahawk as 4096 bytes but it could vary. Each subsequent message contains the next block from the file, and all but the message containing the final block will have the FRAGMENT flag set.

The streaming protocol supports seeking as well. The Tomahawk that is receiving the file can send a message with the payload string "blockN" where N is the index of the block to seek to. For example, to seek to the 12th block in a file (which is 12 * 4096 = 49152 bytes into the file), Tomahawk sends a message with the payload "block12". When the streaming Tomahawk receives the seek message it sends the message "doneblockN" (eg. "doneblock12"), seeks within the file, and begins sending data from there. Both the "blockN" and "doneblockN" messages have the RAW and FRAGMENT flags set.

Retrieved from "http://wiki.tomahawk-player.org/mediawiki/index.php/Network_Protocol"
Personal tools