System input/output

A stream is a sequence of 8-bit bytes. Usually, streams can only be accessed serial, one byte at a time. Bytes flow (are read) from input streams and bytes are sent (written) to output streams.

The System class provides the following final static variables for input and output.

System.in : java.io.InputStream
System.out : java.io.PrintStream
System.err : java.io.PrintStream

System.in reads characters from the keyboard. System.out and System.err send characters to the users terminal window.

The streams can be reassigned with setErr, setIn, and setOut.

Redirecting standard input, output, and error with BASH

A Java program's standard input can be redirected from the user's keyboard to an existing file with:

java Program < data_file

After redirecting the input, any data read with System.in will come from data_file.

Output normally sent to the users screen can be redirected to a file with the command:

java Program > out_file

out_file now contains the output from System.out. If out_file already existed, then it is first truncated.

Output sent to System.err is redirected with:

java Program 2> out_file

Output can be appended to a file from a redirected output with:

java Program >> out_file 
java Program 2>> out_file

java.io.OutputStream

The java.io.OutputStream class provides the following methods:

void close() - flush data to output and close
void flush() - flush data to output
void write( byte[] b) - write bytes from b
void write( byte[] b, int off, int len) - write bytes from b
void write( int b) - write b as a byte

Streams are used to send/receive bytes. A stream does not handle the translation of characters into their local storage encoding.

An output stream can be connected to a printer, to a disk, to a file system, to a network connection.

The I/O streams in Java follow the behaviour of the File I/O in Unix.

java.io.OutputStream example

The OutputStreamTest writes a sequence of bytes to the System.out stream.

OutputStreamTest.java

import java.io.IOException;
import java.io.OutputStream;

public class OutputStreamTest {

    public static void main( String[] args ) {
        for( int i = 48 ; i < 90; i++ ) {
            System.out.write( i );
        }
        System.out.write( 10 );
        System.out.flush();
    }

}

This program outputs:

OutputStreamTest.out

0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXY

Where is the "012..." string in the above program?

java.io.InputStream

The java.io.InputStream class provides the following methods:

int read() - read next byte, returns -1 on EOF
int read( byte[] b) - read bytes into b, returns count
void close() - close the connection to the input source
int available() - number of bytes that can be read

No confusion is caused by returning -1 to indicate an EOF, since a byte can only range in values from 0 to 255.

The input read as bytes must be converted into characters so that a Java program can easily handle the input.

java.io.InputStream test

Bytes read from the standard input are printed as decimal integers.

InputStreamTest.java

import java.io.OutputStream;
import java.io.IOException;

public class InputStreamTest {

    public static void main( String[] args ) throws IOException {
        int b;
        int count = 0;
        while ( (b=System.in.read()) != -1 ) {
            System.out.print( b + " " );
            count++;
            if ( count > 20 ) {
                System.out.println();
                count = 0;
            }
        }
        System.out.println();
    }

}

The throws IOException declarations tells the compiler that it is OK to ignore IOException exceptions.

The InputStreamTest and OutputStreamTest can be tested together with the command:

java OutputStreamTest | java InputStreamTest

The two program are run at the same time, and the output of the first is connected to the input of the second. The output is:

InputStreamTest.out

48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 
69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 
10

Notice that bytes read by InputStreamTest are identically to ones written by OutputStreamTest.

java.io.Reader

The java.io.Reader class provides input as characters. Some of its methods are:

int read() - read next character, returns -1 on EOF
int read( char[] cbuf) - read characters in cbuf, return count
void close() - close the connection to the input source

No confusion is caused by returning -1 to indicate an EOF, since a char can only range in values from 0 to 65535.

The java.io.InputStreamReader converts bytes into characters. The java.io.InputStreamReader class is used to convert System.in to a Reader. Creating a reader for the standard input using the default character encoding is done with:

Reader rd = new InputStreamReader( System.in );

The java.io.InputStreamReader is an example of the adapter design pattern.

Why java.io.Reader and java.io.InputStream

The most common character code in North America is the 7-bit ASCII (American Standard Code of Information Interchange) code. A byte is used to store an ASCII character. A listing of the code is available with the command: man ascii.
Many text files and input/output devices still use ASCII encoding.
Characters in Java use the 16-bit Unicode encoding. Unicode can represent many of the characters found in non-latin alphabets. Unicode enables the creation of programs the handle many languages fairly easily (i.e., it supports i18n (internationalization)).
The main task of the java.io.Reader class is to convert from 8-bit ASCII to 16-bit Unicode. Or more generally, to convert from the local character encoding to unicode.

Conversion with other character encodings

Conversion of bytes to characters using different character sets is done with:


	    InputStreamReader(InputStream in, Charset cs)

Charset is defined in the java.nio package. Java supports the following character sets:

US-ASCII
ISO-8859-1
UTF-8
UTF-16BE
UTF-16LE
UTF-16

A character set is created with:


	    Charset.forName( String name )

Uppercase conversion

The UpperConvert program reads characters from System.in, converts them to uppercase and outputs the characters with System.out.

UpperConvert.java

import java.io.IOException;
import java.io.InputStreamReader;

public class UpperConvert {
    public static void main( String[] args ) throws IOException {
        InputStreamReader rd = new InputStreamReader(System.in);
        int ch = 0;

        while( (ch=rd.read()) != -1 ) {
            char c = (char)ch;
            System.out.print( Character.toUpperCase( c ) );
        }
    }
}

Given the command

java UpperConvert

An the input:

hi there
1 2 3 4 5
bYe

the program will produce:

HI THERE
1 2 3 4 5
BYE

Uppercase conversion with string

Programming using strings is usually more convenient than using characters. The java.io.BufferedReader provides a readLine method that returns the characters in a line as String. The java.io.BufferedReader can get its input from an java.io.InputStreamReader

The uppercase conversion program using strings is:

UpperStringConvert.java

import java.io.IOException;
import java.io.InputStreamReader;
import java.io.BufferedReader;

public class UpperStringConvert {
    public static void main( String[] args ) throws IOException {
        InputStreamReader isd = new InputStreamReader(System.in);
        BufferedReader rd = new BufferedReader( isd );
        String line = null;
        while( (line=rd.readLine()) != null ) {
            System.out.println( line.toUpperCase() );
        }
    }
}

There is very little benefit, in the uppercase conversion examples, but in general the String class provides many more useful operations.

Input from a file

Most of the data that programs work with does not come form the user's keyboard, instead it comes for files. A file is a sequence of bytes managed by the operating system. While a file can be treated as an array, many operating systems only allow read and write access. A file can be treated as a source or destination of characters, the same as input from a user's keyboard and output to a user's terminal window.

Operating system perform the following operations on files:

open
read
write
seek - only some files
close

Associated with each file is a read pointer (the next character to read) and a write pointer, the seek method modifies the read or write pointers.

Reading from a file

The java.io.FileReader class can be used to read information from a text file. A program to read from a file and perform uppercase conversion is:

FileUpperConvert.java

import java.io.IOException;
import java.io.BufferedReader;
import java.io.FileReader;

public class FileUpperConvert {
    public static void main( String[] args ) throws IOException {
        BufferedReader rd =
            new BufferedReader( new FileReader( args[0] ));
        String line = null;
        while( (line=rd.readLine()) != null ) {
            System.out.println( line.toUpperCase() );
        }
        rd.close();
    }
}

The file name is passed in as an argument to the program. The program can be run with

java FileUpperConvert filename

where filename is replaced by the actual file. Note that only the input source for the BufferedReader changes.

Output to a file

The java.io.FileWriter and java.io.BufferedWriter classes can be used to send output to a newly created file.

The program CopyFile copies lines from the input file and saves the lines to an output file. It is almost equivalent to the Unix command:

cp old_file new_file

CopyFile.java

import java.io.IOException;
import java.io.BufferedReader;
import java.io.FileReader;
import java.io.BufferedWriter;
import java.io.FileWriter;

public class CopyFile {
    public static void main( String[] args ) throws IOException {
        if( args.length != 2 ) {
            System.out.println("usage: java CopyFile infile outfile");
            System.exit( 1 );
        }
        BufferedReader rd =
            new BufferedReader( new FileReader( args[0] ));
        BufferedWriter wt =
            new BufferedWriter( new FileWriter( args[1] ));
        String line = null;
        while( (line=rd.readLine()) != null ) {
            wt.write( line );
            wt.newLine();
        }
        rd.close();
        wt.close();
    }
}

It is important to call close on a FileWriter object to ensure that all of the data sent to the file is saved.

Can you think of any problems with the above program if a binary file is copied?

java.io.File

The java.io.File provides information about files and directories. Some of its methods are:

canRead(), canWrite()
delete()
exists()
getName()
isDirectory(), isFile(), isHidden(), isAbsolute()
getName(), getPath()
length()
list(), listFiles()

The DirList class demonstrates how File can be used to list the contents of a directory.

DirList.java

import java.io.File;
import java.io.IOException;

public class DirList {
    public static void main( String[] args ) throws IOException {
        if ( args.length != 1 ) {
            System.out.println("usage: java DirList dir");
            System.exit( 1 );
        }
        String dirname = args[0];
        File dir = new File( dirname );
        if( !dir.isDirectory() ) {
            System.out.println(dirname + " is not a directory");
            System.exit( 1 );
        }
        String[] list = dir.list();
        for( int i = 0 ; i < list.length; i++ ) {
            System.out.println( list[i] );
        }
    }
}

Running the program with java DirList . produces:

DirList.out

FileUpperConvert.java
SumFile.java
DirList.java
CopyFile.java
UpperStringConvert.java
UpperConvert.java
ScannerSum.java
OutputStreamTest.class
OutputStreamTest.java
OutputStreamTest.out
InputStreamTest.java
InputStreamTest.class
InputStreamTest.out
DirList.class

Text vs Binary File

An operating system treats a file as a sequence of bytes. Files that contain characters and only characters are called text files. Text file can be edited and easily displayed to the screen. Any file that does not contain a sequence of characters is called a binary file. Binary files require special application that can examine their contents. PNG, MPG, WAV are all examples of binary files.

The Reader and Writer classes process text files. Binary files can be processed with the DataOutputStream and DataInputStream classes from the java.io package.

Buffering

The efficiency of I/O operations for Readers and Streams can be improved with buffering. The BufferedInputStream class can buffer any class that extends InputStream . A BufferedInputStream is created with:

BufferedInputStream(InputStream in)
BufferedInputStream(InputStream in, int size)

The BufferedInputStream will attempt to read up to size bytes in one read.

An output buffer is created with:

BufferedOutputStream(OutputStream out)
BufferedOutputStream(OutputStream out, int size)

A write only occurs when the buffer contains size bytes or the flush() method is called.

Buffering and Benchmarks

The BufTest program measures the affect of using buffering on a output stream.

BufTest.java

import java.io.OutputStream;
import java.io.BufferedOutputStream;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;

public class BufTest {
    static final int NUM_RECORDS = 128 * 1024;
    static final int REC_SIZE = 64;

    public static void main( String[] args ) throws Exception {
	int bufSize = 4096;
	if ( args.length >= 1 ) {
	    try {
		bufSize = Integer.parseInt( args[0] );
	    }
	    catch( NumberFormatException ex ) {
		// ignore
	    }
	}
	System.out.println("buffer size " + bufSize );

	byte[] buf = new byte[REC_SIZE];
	for( int i = 0 ; i < buf.length; i++ ) buf[i] = (byte)i;

	File saveFile = new File("speed1.dat");
	FileOutputStream save = new FileOutputStream( saveFile );
	long dt = writeTest( buf, save );
	saveFile.delete();
	System.out.println("nonbuffered " + dt );

	File bufFile = new File("speed2.dat");
	BufferedOutputStream buffered =
	    new BufferedOutputStream(
		new FileOutputStream( bufFile ), bufSize );
	dt = writeTest( buf, buffered );
	bufFile.delete();
	System.out.println("buffered " + dt );
    }

    public static long writeTest( byte[] buf, OutputStream out )
	throws IOException
    {
	long t = System.currentTimeMillis();
	for( int i = 0 ; i < NUM_RECORDS; i++ ) {
	    out.write( buf );
	}
	out.close();
	return System.currentTimeMillis() - t;
    }
}

The command, java BufTest, produces:

buffer size 4096
nonbuffered 1637 
buffered 186

Writing to a file on disk

A file is stored on a disk as a sequence of disk blocks. A disk block holds between 512 to 8192 bytes. A 1 byte file and a 511 byte file, both occupy a 512 byte block. Data can only be written or read from a disk as blocks.

For example, if a file contains 800 bytes, the file will occupy two 512 disk blocks. If 100 more bytes are written to the file, then the second block must be read, the bytes are added starting at 288 (800-512) offset into the block, and the block is written back to the disk. A write to a partially full block requires the block to be read.

Buffering of output data to a file can result in large performance gains by eliminating unecessary disk reads and writes.

Advantages/Disadvantages of buffered I/O

Advantages:

Eliminates unecessary disk I/O
Allows more efficient disk I/O by batching operations

Disadvantages:

Increased latency
Increased chance of data loss if the power fails

Benchmarking buffered network communication

Buffering the I/O streams from a socket can also increase the performace. The following code adds buffering and time measurement to the AddClient program.

AddClient.java

import java.io.*;
import java.net.*;

class AddClient {
    public static void main( String[] args ) {
        if ( args.length != 4 ) {
	    System.out.println("usage: java AddClient host port range bufsize");
	    System.exit(0);
        }
        int range = 0;
	int port = 0;
	String host = null;
        int bufSize = 0;
	try {
	    host = args[0];
	    port = Integer.parseInt( args[1] );
            range = Integer.parseInt( args[2] );
            bufSize = Integer.parseInt( args[3] );
	}
	catch( NumberFormatException e ) {
	    System.out.println("bad port number or range");
	    System.exit(0);
	}

        try {
	    /* determine the address of the server and connect to it */
	    InetAddress server = InetAddress.getByName( host );
	    Socket sock = new Socket( server, port );
            OutputStream out = sock.getOutputStream();

            if ( bufSize != 0 ) {
                out = new BufferedOutputStream( out, bufSize );
            }

            DataOutputStream dout = new DataOutputStream( out );
            DataInputStream din =
                new DataInputStream( sock.getInputStream() );

            long t = System.currentTimeMillis();
            /* tx size */
            dout.writeInt( range );
            /* tx ints to add */
            for( int i = 0 ; i < range; i++ ) {
                dout.writeInt( i );
            }
            dout.flush();
            /* retrieve result */
            int result = din.readInt();
            long dt = System.currentTimeMillis() - t;
            System.out.println("result is " + result );
            System.out.println("time is " + dt );
            /* tell the server that we are done */
            dout.writeInt( -1 );
            dout.flush();
	    sock.close();
            dout.close();
            din.close();
	}
	catch( UnknownHostException e ) {
	    System.out.println("bad host name");
	    System.exit(0);
	}
	catch( IOException e ) {
	    System.out.println("io error:" + e);
	    System.exit(0);
	}
    }
}

Benchmarking of the program does demonstrate increased performance.