0004097: LaTeX "can't find file" when file is on a large filesystem because 32-bit stat() call is used

(0008347)
pfelecan (manager)
2010-10-04 18:48

I'm using teTeX daily on a multi-TB ZFS file system and had never this issue. What are the full characteristics of your file system?

(0008348)
pfelecan (manager)
2010-10-04 18:49

the detailed characteristics of the file system where this issue occurs.

(0008351)
adb (reporter)
2010-10-04 21:10

[I sent this earlier via email, but apparently that doesn't update the issue notes]

I think I was mistaken about it being related to # of blocks; it may
actually be a issue with the size of inode numbers.

The filesystem is NFS-mounted from an Isilon cluster. It's currently
1TB; at the time I submitted the bug report, I think it was rather
larger. (It's carved out with a server-side quota from a 1PB
filesystem.) I still see the bug on all our Isilon filesystems, but
not on NetApp or local filesystems.

Files on the Isilon filesystems typically have inode numbers > 2^32,
while creating the test file on the NetApp and local filesystems
generally yields inode numbers in the millions. Attached are example
LaTeX runs with stat information about the file and filesystems.

Diffing the strace outputs (stripped of everything but stat calls,
with pointer values scrubbed to make the diff work), we see

xstat(2, "/opt/csw/share/texmf-dist/aliases", POINTER) = 0
fstat64(3, POINTER) = 0
fstat64(3, POINTER) = 0
-xstat(2, ".", POINTER) = 0
-xstat(2, "./test.ltx", POINTER) = 0
+xstat(2, ".", POINTER) Err#79 EOVERFLOW
fstat64(1, POINTER) = 0
fstat64(1, POINTER) = 0

i.e., in the successful run, we stat the current directory, then stat
the file, but in the failed run we stat the current directory, get
EOVERFLOW, and bail.

The "stat" command uses lstat64() rather than xstat(). (I'm assuming
xstat() is actually getting called behind the scenes by a library
function, since it's undocumented...)

(0008352)
pfelecan (manager)
2010-10-05 09:59

Thank you for the information. I will look for a test case involving kpathsea. When I have something I'll come back.

(0009991)
pfelecan (manager)
2012-06-29 16:51

teTeX is old and I don't plan to release a new package but rather to deliver a texlive based set of packages; use smaller file systems...

Issue History
Date Modified	Username	Field	Change
2009-12-24 02:53	adb	New Issue
2009-12-27 11:36	pfelecan	Status	new => assigned
2009-12-27 11:36	pfelecan	Assigned To	=> pfelecan
2010-10-04 18:48	pfelecan	Note Added: 0008347
2010-10-04 18:49	pfelecan	Note Added: 0008348
2010-10-04 18:49	pfelecan	Status	assigned => feedback
2010-10-04 21:09	adb	File Added: failure.txt
2010-10-04 21:09	adb	File Added: success.txt
2010-10-04 21:10	adb	Note Added: 0008351
2010-10-05 09:59	pfelecan	Note Added: 0008352
2012-06-29 16:51	pfelecan	Note Added: 0009991
2012-06-29 16:51	pfelecan	Status	feedback => closed
2012-06-29 16:51	pfelecan	Resolution	open => won't fix

Notes
(0008347) pfelecan (manager) 2010-10-04 18:48	I'm using teTeX daily on a multi-TB ZFS file system and had never this issue. What are the full characteristics of your file system?

(0008348) pfelecan (manager) 2010-10-04 18:49	the detailed characteristics of the file system where this issue occurs.

(0008351) adb (reporter) 2010-10-04 21:10	[I sent this earlier via email, but apparently that doesn't update the issue notes] I think I was mistaken about it being related to # of blocks; it may actually be a issue with the size of inode numbers. The filesystem is NFS-mounted from an Isilon cluster. It's currently 1TB; at the time I submitted the bug report, I think it was rather larger. (It's carved out with a server-side quota from a 1PB filesystem.) I still see the bug on all our Isilon filesystems, but not on NetApp or local filesystems. Files on the Isilon filesystems typically have inode numbers > 2^32, while creating the test file on the NetApp and local filesystems generally yields inode numbers in the millions. Attached are example LaTeX runs with stat information about the file and filesystems. Diffing the strace outputs (stripped of everything but stat calls, with pointer values scrubbed to make the diff work), we see xstat(2, "/opt/csw/share/texmf-dist/aliases", POINTER) = 0 fstat64(3, POINTER) = 0 fstat64(3, POINTER) = 0 -xstat(2, ".", POINTER) = 0 -xstat(2, "./test.ltx", POINTER) = 0 +xstat(2, ".", POINTER) Err#79 EOVERFLOW fstat64(1, POINTER) = 0 fstat64(1, POINTER) = 0 i.e., in the successful run, we stat the current directory, then stat the file, but in the failed run we stat the current directory, get EOVERFLOW, and bail. The "stat" command uses lstat64() rather than xstat(). (I'm assuming xstat() is actually getting called behind the scenes by a library function, since it's undocumented...)

(0008352) pfelecan (manager) 2010-10-05 09:59	Thank you for the information. I will look for a test case involving kpathsea. When I have something I'll come back.

(0009991) pfelecan (manager) 2012-06-29 16:51	teTeX is old and I don't plan to release a new package but rather to deliver a texlive based set of packages; use smaller file systems...

Relationships

OpenCSW Bug Tracker