Protocol encoding

A message is encoded with text when each byte or byte sequence represents a character from a human language. Many distributed systems use text as their message encoding in their protocol. Text encoding allows non-specialized tools to be used during development.

A message is encoded in binary when the position of the byte in the message stream determines the type of data that the byte encodes. Binary encoding is also common in the implementation of distributed systems.

sending and receiving non-text data

Non-text data can be exchanged with the DataOutputStream and DataInputStream classes. The DataOutputStream class provides:

writeXXX(XXX v)

methods to serialize any basic type and the String class. Examples are:

void writeByte(int v)

void writeDouble(double v)

void writeUTF(String v)

The serialized data can be extracted with:

XXX readXXX()

methods from the DataInputStream class. Examples are:

boolean readBoolean()

float readFloat()

String readUTF()

This class serializes the data in a standard format that works across all platforms.

Data encoding format

Each data type has a defined encoding format.

boolean: true sent as a byte with a value of 1, false sent as a byte with a value of 0.
byte: sent as is.
short: high byte sent, followed by low byte.
char: high byte sent, followed by low byte.
int: big endian, high byte first, followed by the three remaining bytes in decreasing order of significance.
long: high byte first, followed by the 7 remaining bytes.
float: sent as IEEE float floating number.
double: sent as IEEE double floating number.
String: the number of bytes to follow is sent as a short (high byte, low byte), then the remaining bytes of the UTF-8 encoding of the String.

Data output demo

An integer, float, and UTF-8 encoded string is sent to the standard output with:

DataOut.java

import java.io.IOException;
import java.io.DataOutputStream;

class DataOut {
    public static void main( String[] args ) throws IOException {
        DataOutputStream out = new DataOutputStream( System.out );

	out.writeInt( 47 );
	out.writeDouble( 3.1415926 );
	out.writeUTF( "Hello world" );
	out.flush();
	out.close();
    }
}

The output bytes can be examined with:

DataOut.out

% java DataOut > d.out
% od -b d.out
0000000 000 000 000 057 100 011 041 373 115 022 330 112 000 013 110 145
0000020 154 154 157 040 167 157 162 154 144
0000031

Data input demo

A matching program that reads an integer, float, and UTF-8 string is:

DataIn.java

import java.io.IOException;
import java.io.DataInputStream;

class DataIn {
    public static void main( String[] args ) throws IOException {
        DataInputStream in = new DataInputStream( System.in );

	System.out.println("int = " + in.readInt() );
	System.out.println("double = " + in.readDouble() );
	System.out.println("chars = " + in.readUTF() );
	in.close();
    }
}

The output produced by DataIn when feed data created by DataOut is:

DataIn.out

% java DataIn < d.out
int = 47
double = 3.1415926
chars = Hello world

client/server that adds ints

The AddServer will add a sequence of integers sent by a AddClient and return the result. These programs illustrate sending non-text data.

The data exchanged after the client connects is:

Client:: size int0 int1 ...
Server:: sum

Every integer is sent with:

writeInt( int v )

and extracted with:

int readInt()

Communication can be more efficient when the direct encoding of integers are sent and received. Sending an int requires only 4 bytes.

What are the disadvantages of sending binary as apposed to text data?

AddServer

Wait for clients to send a sequence of integers to sum, and reply with the sum. The connection to the client is terminated if a zero sized sequence is sent. This is indicated by a -1 in the size field.

AddServer.java

import java.io.*;
import java.net.*;

class AddServer {
    public static void main( String[] args ) {
	try {
	    ServerSocket listen = new ServerSocket( 0 );
	    System.out.println("Server port is " + listen.getLocalPort() );

            /* handle one client at a time */
	    while ( true ) {
		Socket sock = listen.accept();

                /* data from client */
		DataInputStream din =
                    new DataInputStream( sock.getInputStream() );

                /* data to client */
		DataOutputStream dout =
                    new DataOutputStream( sock.getOutputStream() );

                for(;;) {
                    int sum = 0;
                    int sz = din.readInt();
                    if ( sz <= 0 ) break;
                    for( int i = 0 ; i < sz; i++ ) {
                        sum += din.readInt();
                    }
                    dout.writeInt( sum );
                    dout.flush();
                }
		System.out.println("Connection closed");
		sock.close(); // clean up required
                din.close();
                dout.close();
	    }
	}
	catch( IOException e ) {
	    System.out.println("error: " + e );
	}
    }
}

AddClient

AddClient is a test client for AddServer, it sends the sequence 1 .. n to sum.

AddClient.java

import java.io.*;
import java.net.*;

class AddClient {
    public static void main( String[] args ) {
        int range = 0;
	int port = 0;
	String host = null;
	try {
	    host = args[0];
	    port = Integer.parseInt( args[1] );
            range = Integer.parseInt( args[2] );
	}
	catch( NumberFormatException e ) {
	    System.out.println("bad port number or range");
	    System.exit(0);
	}
	catch( IndexOutOfBoundsException e ) {
	    System.out.println("usage: java AddClient host port range");
	    System.exit(0);
	}

        try {
	    /* determine the address of the server and connect to it */
	    InetAddress server = InetAddress.getByName( host );
	    Socket sock = new Socket( server, port );
            DataInputStream din =
                new DataInputStream( sock.getInputStream() );
            DataOutputStream dout =
                new DataOutputStream( sock.getOutputStream() );

            /* tx size */
            dout.writeInt( range );
            /* tx ints to add */
            for( int i = 0 ; i < range; i++ ) {
                dout.writeInt( i );
            }
            dout.flush();
            /* retrieve result */
            int result = din.readInt();
            System.out.println("result is " + result );
            /* tell the server that we are done */
            dout.writeInt( -1 );
            dout.flush();
	    sock.close();
            dout.close();
            din.close();
	}
	catch( UnknownHostException e ) {
	    System.out.println("bad host name");
	    System.exit(0);
	}
	catch( IOException e ) {
	    System.out.println("io error:" + e);
	    System.exit(0);
	}
    }
}

Calculator client/server

The calculator server accepts requests to add, subtract, multiple, and divide a pair of integers. The previous two application perform only one type of request from the clients. A more complex protocol is required to accommodate the calculator requests.

The messages accepted by the server are:

ADD: 1 int0 int1
MULT: 2 int0 int1
SUBTRACT: 3 int0 int1
DIVIDE: 4 int0 int1
EXIT: 5

If it is an arithmetic request, the server computes the integer result and sends that result. An EXIT request causes the server to drop the connection.

protocol header

To ensure that the message types are the same between the client and server, the CalculatorCommands interface is used:

CalculatorCommands.java

/* a message contains the command and two possible arguments,
 * all the values are sent as ints */
interface CalculatorCommands {
    final static int ADD = 1;
    final static int MULT = 2;
    final static int SUBTRACT = 3;
    final static int DIVIDE = 4;
    final static int EXIT = 5;
}

Note: once the protocol is specified then multiple distinct clients and servers can be implemented.

Calculator server

The server decodes the incoming messages, and carries out the request by calling a routine that handles the request. Decoding and calling the relevant routine is often called dispatching.

CalculateServer.java

import java.io.*;
import java.net.*;

class CalculateServer {
    private ServerSocket listener;
    private CalculateLogger logger;

    public CalculateServer( int port ) throws IOException {
	/* determine the address of the server and connect to it */
	listener = new ServerSocket( port );
        logger = new CalculateLogger("calc.log");
    }

    public void serve() throws IOException {
        while ( true ) {
	    Socket sock = listener.accept();
            logger.report("accept: " + sock );
	    try {
		Calculator calc =
                    new Calculator(
                        sock.getInputStream(),
                        sock.getOutputStream() );
		calc.run();
	    }
	    catch( IOException  e ) {
                throw new RuntimeException( e );
	    }
	    sock.close();
            logger.report("close: " + sock );
	}
    }

    public static void main( String[] args ) {
	int port = 0;
	String host = null;
	try {
	    port = Integer.parseInt( args[0] );
	}
	catch( NumberFormatException e ) {
	    System.out.println("bad port number");
	    System.exit(0);
	}
	catch( IndexOutOfBoundsException e ) {
	    System.out.println("usage: java CalculateServer port");
	    System.exit(0);
	}

	try {
	    CalculateServer server = new CalculateServer( port );
	    server.serve();
	}
	catch( IOException e ) {
	    System.out.println("io error:" + e);
	    System.exit(0);
	}
    }
}

class Calculator {
    private DataInputStream requests;
    private DataOutputStream replys;
    
    public Calculator( InputStream in, OutputStream out ) {
	requests = new DataInputStream( new BufferedInputStream( in ));
	replys = new DataOutputStream( new BufferedOutputStream( out ));
    }

    public void run() {
	try {
	    while( true ) {
		int cmd = requests.readInt();
		if ( cmd == CalculatorCommands.EXIT ) {
		    break; 
		}
		else if ( cmd == CalculatorCommands.ADD ) {
		    replys.writeInt(
                        requests.readInt() + requests.readInt() );
		    replys.flush();
		}
		else if ( cmd == CalculatorCommands.MULT ) {
		    replys.writeInt(
                        requests.readInt() * requests.readInt() );
		    replys.flush();
		}
		else if ( cmd == CalculatorCommands.SUBTRACT ) {
		    replys.writeInt(
                        requests.readInt() - requests.readInt() );
		    replys.flush();
		}
		else if ( cmd == CalculatorCommands.DIVIDE ) {
		    replys.writeInt(
                        requests.readInt() / requests.readInt() );
		    replys.flush();
		}
		else {
		    // unkown command, give up
		    break;
		}
	    }
	    requests.close();
	    replys.close();
	}
	catch( IOException e ) {
	    // just give up right now
            throw new RuntimeException( e );
	}
    }
}

logging

The status and activity of server programs are often monitored with the aid of log files. A log file is a record of the activity of some program. All server program will typically have a log file. The Java API provides the java.util.logging package.

CalculateLogger.java

import java.io.*;

public class CalculateLogger {
    private BufferedWriter logfile;
    private String filename;

    public CalculateLogger( String filename)
        throws IOException
    {
        this.filename = filename;
        this.logfile =
            new BufferedWriter( new FileWriter( filename ));
    }

    public void report( String s ) {
        try {
            logfile.write( s + "\n" );
            logfile.flush();
        }
        catch( IOException ex ) {
            // ignore logging errors
        }
    }

    public void close() {
        try {
            logfile.close();
        }
        catch( IOException ex ) {
            // ignore logging errors
        }
    }
}

Calculator client

The calculator client is responsible for providing a user interface. The client accepts users commends, encodes and sends the request to the server. The reply from the server is then output to the user.

CalculateClient.java

import java.io.*;
import java.net.*;
import java.util.StringTokenizer;

class CalculateClient {
    private InetAddress serverHost;
    private int serverPort;
    private Socket calculate;
    private DataInputStream reply;
    private DataOutputStream request;

    public CalculateClient( String host, int port ) throws IOException {
	/* determine the address of the server and connect to it */
	serverHost = InetAddress.getByName( host );
	serverPort = port;
	calculate = new Socket( serverHost, serverPort );

	/* get the output stream */
	OutputStream out = calculate.getOutputStream();
	request = new DataOutputStream( new BufferedOutputStream( out ));

	/* get the input stream */
	InputStream in = calculate.getInputStream();
	reply = new DataInputStream( new BufferedInputStream( in ));

    }

    public void handleUserInput() throws IOException {
	BufferedReader user =
            new BufferedReader( 
                new InputStreamReader( System.in ) );

	System.out.print("calc: ");
	String line = null;
        while( (line=user.readLine()) != null ) {
	    StringTokenizer parse = new StringTokenizer( line );
	    if ( !parse.hasMoreTokens() ) continue;
	    String cmd = parse.nextToken();
	    if ( cmd.equals("add") ) {
		int a = Integer.parseInt( parse.nextToken() );
		int b = Integer.parseInt( parse.nextToken() );
		request.writeInt( CalculatorCommands.ADD );
		request.writeInt( a );
		request.writeInt( b );
		request.flush();
		System.out.println( "result = " + reply.readInt() );
	    }
	    else if ( cmd.equals("mult") ) {
		int a = Integer.parseInt( parse.nextToken() );
		int b = Integer.parseInt( parse.nextToken() );
		request.writeInt( CalculatorCommands.MULT );
		request.writeInt( a );
		request.writeInt( b );
		request.flush();
		System.out.println( "result = " + reply.readInt() );
	    }
	    else if ( cmd.equals("sub") ) {
		int a = Integer.parseInt( parse.nextToken() );
		int b = Integer.parseInt( parse.nextToken() );
		request.writeInt( CalculatorCommands.SUBTRACT );
		request.writeInt( a );
		request.writeInt( b );
		request.flush();
		System.out.println( "result = " + reply.readInt() );
	    }
	    else if ( cmd.equals("divide") ) {
		int a = Integer.parseInt( parse.nextToken() );
		int b = Integer.parseInt( parse.nextToken() );
		request.writeInt( CalculatorCommands.DIVIDE );
		request.writeInt( a );
		request.writeInt( b );
		request.flush();
		System.out.println( "result = " + reply.readInt() );
	    }
	    else if ( cmd.equals("exit") ) {
		request.writeInt( CalculatorCommands.EXIT );
		request.flush();
		break; // exit
	    }
	    System.out.print("calc: ");
	}
	request.close();
	reply.close();
    }


    public static void main( String[] args ) {
	int port = 0;
	String host = null;
	try {
	    host = args[0];
	    port = Integer.parseInt( args[1] );
	}
	catch( NumberFormatException e ) {
	    System.out.println("bad port number");
	    System.exit(0);
	}
	catch( IndexOutOfBoundsException e ) {
	    System.out.println("usage: cmd host port");
	    System.exit(0);
	}

	try {
	    CalculateClient calc = new CalculateClient( host, port );
	    calc.handleUserInput();
	}
	catch( UnknownHostException e ) {
	    System.out.println("bad host name");
	    System.exit(0);
	}
	catch( IOException e ) {
	    System.out.println("io error:" + e);
	    System.exit(0);
	}
    }
}

How cohesive is the handleUserInput() method?

Refactored client

We can improve the design of the client, by moving the command transmission and retrieval of the result into separate methods. These methods hide the network communication.

CalculateClientRefactored.java

import java.io.*;
import java.net.*;
import java.util.StringTokenizer;

class CalculateClientRefactored {
    private InetAddress serverHost;
    private int serverPort;
    private Socket calculate;
    private DataInputStream reply;
    private DataOutputStream request;

    public CalculateClientRefactored(
            String host, int port )
        throws IOException
    {
	/* determine the address of the server and connect to it */
	serverHost = InetAddress.getByName( host );
	serverPort = port;
	calculate = new Socket( serverHost, serverPort );

	/* get the output stream */
	OutputStream out = calculate.getOutputStream();
	request = new DataOutputStream( new BufferedOutputStream( out ));

	/* get the input stream */
	InputStream in = calculate.getInputStream();
	reply = new DataInputStream( new BufferedInputStream( in ));
    }

    private int doAdd( int a, int b ) throws IOException {
        request.writeInt( CalculatorCommands.ADD );
        request.writeInt( a );
        request.writeInt( b );
        request.flush();
        return reply.readInt();
    }

    private int doSub( int a, int b ) throws IOException {
        request.writeInt( CalculatorCommands.SUBTRACT );
        request.writeInt( a );
        request.writeInt( b );
        request.flush();
        return reply.readInt();
    }

    private int doMult( int a, int b ) throws IOException {
        request.writeInt( CalculatorCommands.MULT );
        request.writeInt( a );
        request.writeInt( b );
        request.flush();
        return reply.readInt();
    }

    private int doDivide( int a, int b ) throws IOException {
        request.writeInt( CalculatorCommands.DIVIDE );
        request.writeInt( a );
        request.writeInt( b );
        request.flush();
        return reply.readInt();
    }

    private void doExit() throws IOException {
        request.writeInt( CalculatorCommands.EXIT );
        request.flush();
    }

    public void handleUserInput() throws IOException {
	BufferedReader user =
            new BufferedReader( 
                new InputStreamReader( System.in ) );

	System.out.print("calc: ");
	String line = null;
        while( (line=user.readLine()) != null ) {
	    StringTokenizer parse = new StringTokenizer( line );
	    if ( !parse.hasMoreTokens() ) continue;
	    String cmd = parse.nextToken();
	    if ( cmd.equals("add") ) {
		int a = Integer.parseInt( parse.nextToken() );
		int b = Integer.parseInt( parse.nextToken() );
		System.out.println( "result = " + doAdd(a,b) );
	    }
	    else if ( cmd.equals("mult") ) {
		int a = Integer.parseInt( parse.nextToken() );
		int b = Integer.parseInt( parse.nextToken() );
		System.out.println( "result = " + doMult(a,b) );
	    }
	    else if ( cmd.equals("sub") ) {
		int a = Integer.parseInt( parse.nextToken() );
		int b = Integer.parseInt( parse.nextToken() );
		System.out.println( "result = " + doSub(a,b) );
	    }
	    else if ( cmd.equals("divide") ) {
		int a = Integer.parseInt( parse.nextToken() );
		int b = Integer.parseInt( parse.nextToken() );
		System.out.println( "result = " + doDivide(a,b) );
	    }
	    else if ( cmd.equals("exit") ) {
                doExit();
		break; // exit
	    }
	    System.out.print("calc: ");
	}
	request.close();
	reply.close();
    }


    public static void main( String[] args ) {
	int port = 0;
	String host = null;
	try {
	    host = args[0];
	    port = Integer.parseInt( args[1] );
	}
	catch( NumberFormatException e ) {
	    System.out.println("bad port number");
	    System.exit(0);
	}
	catch( IndexOutOfBoundsException e ) {
	    System.out.println("usage: cmd host port");
	    System.exit(0);
	}

	try {
	    CalculateClient calc = new CalculateClient( host, port );
	    calc.handleUserInput();
	}
	catch( UnknownHostException e ) {
	    System.out.println("bad host name");
	    System.exit(0);
	}
	catch( IOException e ) {
	    System.out.println("io error:" + e);
	    System.exit(0);
	}
    }
}

datagrams receiver

A DatagramSocket can send and receive datagrams to multiply applications. A test program to illustrate receiving datagrams follows:

DatagramServ.java

import java.io.*;
import java.net.*;

public class DatagramServ {
    public static void main( String[] args ) throws IOException{
        DatagramSocket ds;
        DatagramPacket dp;
        InetAddress ia;
        byte[] buf = new byte[1024];

        ds = new DatagramSocket();
        System.out.println("port = " + ds.getLocalPort() );

        /* set timeout to 5000 milliseconds (5 seconds) */
        ds.setSoTimeout( 5000 );

        while ( true ) {
            dp = new DatagramPacket( buf, buf.length );
            try {
                ds.receive( dp );
                int len = dp.getLength();
                System.out.println("rx packet: len=" + len );
                ia = dp.getAddress();
                System.out.println(
                    "from=(" + ia.getHostName() + ","
                    + dp.getPort() + ") text=" + new String(buf,0, len ) );
            }
            catch ( InterruptedIOException e ) {
                System.out.println("timed out");
            }    
        }
    }
}

datagrams transmitter

Demonstrate the transmission of datagrams. Notice that a datagram contains the data and the destination address.

DatagramClnt.java

import java.io.*;
import java.net.*;

public class DatagramClnt {
    public static void main( String[] args ) throws IOException{
        BufferedReader stdin = new BufferedReader(
                 new InputStreamReader( System.in ) );
        DatagramSocket ds; DatagramPacket dp;
        InetAddress ia; int port; byte[] buf;

        ia = InetAddress.getByName( args[0] );
        port = Integer.parseInt( args[1] );
        ds = new DatagramSocket();

        String line = null;
        while ( (line=stdin.readLine()) != null ) {
            buf = line.getBytes();
            dp =  new DatagramPacket( buf, buf.length, ia, port );
            ds.send( dp );
        }
    }
}