Write to Binary files in Java

Question

I am trying to write a lot of data into a binary file. Because it is a lot of data, it is important that this is done fast and I want to be able to write the data as ints one by one. I have tried RandomAccessFile, BufferedWriter, DataOutputStream etc. but all of those are either too slow or cannot write ints. Any ideas that might help me?

Get a faster hard drive ;-) - seriously, how do you know the bottleneck is on the Java side? If the disk is slow, there is nothing Java can do ... — meriton
– meriton, Commented Sep 9, 2012 at 17:07
FYI, the Java team in working on re-inventing Java Serialization. See presentation by Viktor Klang, Marshalling: Data-Oriented Serialization, summer 2025. — Basil Bourque
– Basil Bourque, Commented Sep 25 at 19:55

bmargulies · Accepted Answer · 2012-09-09 16:54:38Z

3

Every stream can 'write ints' if you write the correct code to convert ints to bytes.

The two 'fast' IO options in Java are BufferedOutputStream on top of FileOutputStream and the use of a FileChannel with NIO buffers.

If all you are writing is many, many, int values, you can use IntBuffer instances to pass the data to a fileChannel.

Further, 'one at a time' is generally incompatible with 'fast'. Sooner or later, data has to travel to the disk in blocks. If you force data to disk in small quantities, you will find that the process is very slow. You could, for example, add integer values to a buffer and write the buffer when it fills, and then repeat.

answered Sep 9, 2012 at 16:54

bmargulies

101k40 gold badges196 silver badges328 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Jen Over a year ago

Thanks! I've tried BufferedOutPutstream together with FileOutPutStream, but didn't get how to use intbuffer. I need to look further into it!

bmargulies Over a year ago

You might want to get your shift key looked at. It seems to have a mind of its own.

Dan D. · Accepted Answer · 2012-09-09 16:54:29Z

0

Take a look at the java.nio package. You will find classes that you can use for your needed purposes.

answered Sep 9, 2012 at 16:54

Dan D.

32.4k6 gold badges66 silver badges81 bronze badges

Comments

Dark Squirtings · Accepted Answer · 2012-09-09 17:03:35Z

0

Well, writing to a file one int at a time isn't an inherently fast operation. Even with bufferedwriter you're potentially making a lot of function calls (and may still be doing a lot of file writes if you haven't set the buffer to be large enough).

Have you tried putting the integers into an array, using ByteBuffer to convert it to a byte array, and then writing the byte array to a file?

answered Sep 9, 2012 at 17:03

Dark Squirtings

1988 bronze badges

Collectives™ on Stack Overflow

Write to Binary files in Java

3 Answers 3

2 Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

2 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related