Compare contents of pdf in Java

I am writing Java unit tests and want to compare the contents of a generated pdf to one saved as a resource. I thought I'd be able to read the contents of both into byte arrays and compare, but I'm not able to get the byte arrays to match.

I've tried comparing two pdfs where one is a copy of the other and still no dice, even if I set the createTime, lastModifiedTime, and lastAccessTime of both pdfs.

After using diff checker, I think the problem is how I'm setting the metadata in code. On my laptop, the createTime, modifiedTime, etc are the same in both files I'm trying to compare, but in the diff checker the only difference is the update time and create time, which have not been updated from their initial values.

Does anyone know how to compare the contents of a binary file without considering metadata?

Here is my code, for reference

File actualFile = new File("/tmp/" + pdfName);
Files.setLastModifiedTime(actualFile.toPath(), FileTime.fromMillis(testDateInMillis));
Files.setAttribute(actualFile.toPath(), "creationTime", FileTime.fromMillis(testDateInMillis));
Files.setAttribute(actualFile.toPath(), "lastAccessTime", FileTime.fromMillis(testDateInMillis));
File expectedFile = new File("src/test/resources/test-pdfs/" + pdfName);
Files.setAttribute(expectedFile.toPath(), "creationTime", FileTime.fromMillis(testDateInMillis));
Files.setAttribute(expectedFile.toPath(), "lastAccessTime", FileTime.fromMillis(testDateInMillis));
Files.setLastModifiedTime(expectedFile.toPath(), FileTime.fromMillis(testDateInMillis));

byte[] filebytes = Files.readAllBytes(expectedFile.toPath());
byte[] filebytes2 = Files.readAllBytes(actualFile.toPath());
Assertions.assertEquals(filebytes, filebytes2);