Release 2.1.0Java 8+BSD 3-Clause
Fast, zero-dependency CRC library for Java
Every CRC from CRC-3 to CRC-64 of the RevEng catalogue — CRC-32, CRC-32C, CRC-16/MODBUS, CRC-64/XZ and 108 more — and any custom one up to 64 bits.
byte[] data = "123456789".getBytes(StandardCharsets.US_ASCII);
long modbus = CrcModel.of("CRC-16/MODBUS").value(data); // 0x4B37
long crc32c = CrcModel.of("CRC-32C").value(data); // 0xE3069283
long xz = CrcModel.of("CRC-64/XZ").value(data); // 0x995DC9BBDF1939FA
- 112
- algorithms of the catalogue
- 186
- names with aliases
- 3…64
- bits, 21 widths predefined
- 0
- runtime dependencies
- 8…25
- JDK versions built and tested
Install
The library is on Maven Central: one jar under 70 KB, no dependencies at run time.
<dependency>
<groupId>com.github.sftwnd.crayfish.common</groupId>
<artifactId>crayfish-common-crc</artifactId>
<version>2.1.0</version>
</dependency>
implementation 'com.github.sftwnd.crayfish.common:crayfish-common-crc:2.1.0'
implementation("com.github.sftwnd.crayfish.common:crayfish-common-crc:2.1.0")
On Java 9+ the jar is the named module com.github.sftwnd.crayfish_common_crc; OSGi bundle
headers are included. Coming from 1.x? See
Migrating from 1.x.
What it does
More than a table of polynomials: a CRC is a value you can keep, combine, resume and send.
-
Fast by default
Table-driven slicing-by-8 for every model, not only for CRC-32: 8 bytes per step, 2–4 times faster than one byte per step on buffers from 256 bytes. A CRC up to 32 bits wide is calculated in an
intregister. -
CPU instructions on request
CRC-32, CRC-32/JAMCRC and CRC-32C can be calculated by
java.util.zip.CRC32andCRC32C, which the JVM runs on the CRC instructions of the CPU — with the whole API of the library. -
Immutable CRCs
A CRC is a value: an update or a combination gives a new CRC, so a CRC can be shared between threads and kept as the state of a calculation.
-
Combine and resume
Join the CRCs of parts calculated in parallel threads or on other nodes, like zlib
crc32_combine. Continue from a saved value and length, also of 2 GiB and more. -
Streams
A
java.util.zip.Checksumof any model forCheckedInputStream,CheckedOutputStreamand other code written forCRC32; the CRC of anInputStreamor a channel read to its end. -
Codewords
Build and check data followed by its CRC, in the byte order of the model or of a format such as PNG. The bytes of a CRC in the byte order of a protocol, and its hexadecimal digits.
-
CRCs of bits
The CRC of data that is not a whole number of bytes, such as the frames of CAN, FlexRay and USB.
-
Find the algorithm
Give samples of data with their CRCs and get the algorithms of the catalogue that produce them — to find out the CRC of a device or of a file format.
-
Names and aliases
A model is chosen by the name or by any alias of the catalogue, in any case of its letters:
CRC-32,PKZIPandCRC-32/XZare one model. -
Any CRC up to 64 bits
Width 1…64, any poly, init, refin, refout and xorout: by the parameters, with a builder or from a parameter line of the catalogue.
-
Memory on demand
Lookup tables are built on the first calculation and collected when no model uses them any more. No thread of its own, so the library is unloaded with a redeployed application.
-
Java 8+ and modular
Runs on Java 8; on Java 9+ it is a named JPMS module of a multi-release jar, with OSGi bundle headers. Two builds of the same sources give the same jar byte for byte.
Verified. Every predefined model is tested against its RevEng check value and residue and against a bit-by-bit reference implementation.
In code
The whole API is a model, a CRC and a checksum. The examples are those of the README, which explains every one of them.
The value of a CRC, an immutable CRC to continue or to combine, its hexadecimal digits and
its bytes in the byte order of a protocol.
More
CrcModel model = CrcModel.of("CRC-16/ARC");
long value = model.value(data); // the CRC value only, no CRC object
long part = model.value(data, offset, length); // a part of the data
CRC crc = model.crc().update(part1).update(part2, length2); // part1 followed by the first length2 bytes of part2
String crc32 = CrcModel.of("CRC-32").crc(data).hex(); // cbf43926 for "123456789"
byte[] modbus = CrcModel.of("CRC-16/MODBUS").crc(data).bytes(ByteOrder.LITTLE_ENDIAN); // 37 4b (0x4b37)
The CRC of parts calculated independently. The cost does not depend on the size of the data. More
CRC first = model.crc(part1); // calculated independently,
CRC second = model.crc(part2); // for example in parallel threads
CRC both = first.combine(second); // the CRC of part1 followed by part2
// only the values are kept, like zlib crc32_combine(crc1, crc2, len2):
long value = model.combine(first.value(), second.value(), second.length());
Continue a calculation from a saved CRC value and length, as a CRC or as
a java.util.zip.Checksum.
More
long savedValue = crc.value();
long savedLength = crc.length();
// later, also in another process
CRC resumed = model.crc(savedValue, savedLength).update(moreData);
// append to a file whose CRC is kept
CrcChecksum checksum = model.crc(savedValue, savedLength).checksum();
try (OutputStream file = Files.newOutputStream(path, StandardOpenOption.APPEND);
CheckedOutputStream out = new CheckedOutputStream(file, checksum)) {
out.write(moreData); // after the bytes written before
}
CRC whole = checksum.crc(); // the CRC of the whole file
Any CRC works with the JDK classes made for CRC32; a stream or a channel needs no loop.
More
CrcModel model = CrcModel.of("CRC-32C");
try (CheckedOutputStream out = new CheckedOutputStream(Files.newOutputStream(path), model.checksum())) {
out.write(header); // the parts in the order they are written
out.write(body, 0, bodyLength);
long value = out.getChecksum().getValue(); // the CRC value of all bytes written
}
try (InputStream in = Files.newInputStream(path)) {
CRC crc = model.crc(in); // all the bytes of the stream
}
try (FileChannel channel = FileChannel.open(path)) {
CRC crc = model.crc(header).update(channel); // the header followed by all the bytes of the channel
}
A codeword is data followed by its CRC, as a frame of a protocol or a block of a file carries them. More
CrcModel modbus = CrcModel.of("CRC-16/MODBUS");
byte[] frame = modbus.codeword(pdu); // the PDU followed by its CRC, low byte first
boolean valid = modbus.isValid(frame); // true
boolean inside = modbus.isValid(buffer, 4, 10); // a codeword of 10 bytes at the offset 4 of a buffer
// PNG stores the CRC-32 of a chunk most significant byte first
CrcModel crc32 = CrcModel.of("CRC-32");
byte[] type = "IEND".getBytes(StandardCharsets.US_ASCII);
byte[] iend = crc32.codeword(type, ByteOrder.BIG_ENDIAN); // 49 45 4e 44 ae 42 60 82
boolean chunk = crc32.isValid(data, 4, n + 8, ByteOrder.BIG_ENDIAN); // the type, n bytes of data and the CRC
Some protocols calculate the CRC of bits that are not a whole number of bytes. More
// CAN: the 19 bits before the data and the data, packed most significant bit first
long can = CrcModel.of("CRC-15/CAN").valueOfBits(frameBits, 19 + 8 * dlc);
// FlexRay: the 20 bits of the header in a long
long header = CrcModel.of("CRC-11/FLEXRAY").valueOfBits(headerBits, 20);
// USB: the 11 bits of a token, the first bit in bit 0 (refin)
long token = CrcModel.of("CRC-5/USB").valueOfBits(address | endpoint << 7, 11);
A CRC outside the catalogue: by its parameters, with a builder or from a parameter line. More
CrcModel other = CrcModel.of(16, 0x1234, 0x0, true, true, 0x0); // CRC-16/P1234_I0_RI_RO
CrcModel crc32 = CrcModel.builder(32) // the width
.poly(0x04C11DB7L) // required
.reflected(true) // required: refin and refout, or refin(...) and refout(...)
.init(0xFFFFFFFFL) // 0 if not set
.xorout(0xFFFFFFFFL) // 0 if not set
.check(0xCBF43926L) // optional: build() verifies it
.build(); // CRC-32/ISO-HDLC
CrcParameters arc = CrcParameters.parse("width=16 poly=0x8005 init=0x0000 refin=true refout=true xorout=0x0000");
Which CRC does a device or a file format use? Take samples of its data with their CRCs. More
Set<StandardCrc> found = StandardCrc.matching(data, 0x4B37L); // the algorithms that give the value
found.retainAll(StandardCrc.matching(frame, 0, frame.length - 2, crcOfFrame)); // another sample narrows them down
if (found.size() == 1) { // the algorithm of the samples
CrcModel model = found.iterator().next().model();
}
CRC-32 and CRC-32C on the CRC instructions of the CPU, chosen per model or by a default:
-Dcom.github.sftwnd.crayfish.common.crc.jdk=true.
More
CrcModel jdk = CrcModel.of("CRC-32", true); // java.util.zip.CRC32
CrcModel tables = CrcModel.of("CRC-32", false); // the lookup tables of the library
CrcModel same = tables.withJdk(true); // the model jdk
CrcDescription.setJdk(true); // the default of the models chosen without a choice
CrcModel byDefault = CrcModel.of("CRC-32C"); // java.util.zip.CRC32C on Java 9+
The catalogue, by name
112 algorithms of 21 widths. Every name and alias is accepted by
CrcModel.of(). Pick one to see its parameters and to calculate a CRC right here.
CRC-3 2
CRC-4 2
CRC-5 3
CRC-6 5
CRC-7 3
CRC-8 20
CRC-10 3
CRC-11 2
CRC-12 4
CRC-13 1
CRC-14 2
CRC-15 2
CRC-16 31
CRC-17 1
CRC-21 1
CRC-24 8
CRC-30 1
CRC-31 1
CRC-32 12
CRC-40 1
CRC-64 7
CRC-82/DARC of the catalogue is not predefined: a CRC value is a long, the width
is limited to 64 bits.
Speed
Measured with JMH. The full comparison with 1.2.0 and 2.0.0 on JDK 8, 21 and 25 is on the wiki page Speed of 2.1.0.
- 2–4×slicing-by-8 against one byte per step, on buffers from 256 bytes
- 1.1–1.7×2.1.0 against 2.0 for CRCs up to 32 bits wide: the register of an
int - 16×CRC-32C of 4 KB on CPU instructions through the JDK: 0.13 µs instead of 2.14 µs
| Model, JDK | 16 B | 256 B | 4 KB | 1 MB |
|---|---|---|---|---|
| CRC-32, JDK 25 | 13.8 → 16.0 ns | 141 → 62 ns | 2.24 → 0.20 µs | 515 → 42 µs |
| CRC-32C, JDK 25 | 13.8 → 5.5 ns | 145 → 13 ns | 2.14 → 0.13 µs | 513 → 41 µs |
| CRC-32, JDK 8 | 18.8 → 19.6 ns | 141 → 59 ns | 2.41 → 0.20 µs | 615 → 39 µs |
The lookup tables are the default; the classes of the JDK are chosen per model or by a system property. The details, and what the speed depends on, are in the README: Speed and CRC-32 and CRC-32C of the JDK.
Add it to your build
com.github.sftwnd.crayfish.common:crayfish-common-crc:2.1.0