How to Solve the Fortran Runtime Error: I/O Past End of Record on Unformatted Files

Discover the solution to the `Fortran runtime error` when reading unformatted files. Learn why the error occurs and how to successfully read large binary files in Fortran.
---
This video is based on the question https://stackoverflow.com/q/63868616/ asked by the user 'Stephen Wong' ( https://stackoverflow.com/u/10486963/ ) and on the answer https://stackoverflow.com/a/63930080/ provided by the user 'chw21' ( https://stackoverflow.com/u/4621823/ ) at 'Stack Overflow' website. Thanks to these great users and Stackexchange community for their contributions.

Visit these links for original content and any more details, such as alternate solutions, latest updates/developments on topic, comments, revision history etc. For example, the original title of the Question was: How to solve 'Fortran runtime error: I/O past end of record on unformatted file'?

Also, Content (except music) licensed under CC BY-SA https://meta.stackexchange.com/help/l...
The original Question post is licensed under the 'CC BY-SA 4.0' ( https://creativecommons.org/licenses/... ) license, and the original Answer post is licensed under the 'CC BY-SA 4.0' ( https://creativecommons.org/licenses/... ) license.

If anything seems off to you, please feel free to write me at vlogize [AT] gmail [DOT] com.
---
Understanding the Fortran Runtime Error: I/O Past End of Record

If you have encountered the frustrating Fortran runtime error: I/O past end of record while attempting to read a binary file in Fortran, you're not alone. This error typically signals a discrepancy between how the file was created and how Fortran expects to read it. In this post, we’ll take a closer look at this error, why it occurs, and how to resolve it.

The Problem: Encountering the Runtime Error

The issue arises when reading a large binary file that was generated in Python using the numpy library. Here’s a brief overview of the scenario:

Data Generation: A massive 3D array (1024 x 1024 x 1024) is created and saved as a binary file (filename.dat) using Python.

Testing in Python: Reading this file with Python works perfectly.

Testing in Fortran: When the same file is accessed using a Fortran program, it throws an I/O past end of record error.

The Fortran Code That Causes the Error

Here’s a simplified version of the Fortran code that leads to the error:

[[See Video to Reveal this Text or Code Snippet]]

The runtime error arises during the Read(10) dens statement, particularly for indices greater than 632.

Analyzing the Root Cause

The core of the issue lies in how unformatted files work in Fortran. Here are a few key points to consider:

Unformatted Files: When using form="unformatted", Fortran expects the file to have a specific structure, typically including record headers and footers.

Data Layout: The data layout (how numbers are organized in binary form) generated by Python’s numpy may differ from Fortran's expectations. The numpy.asfortranarray() function ensures the array is in column-major order, but it does not affect how data is actually written to the file.

Comparing Python and Fortran File Outputs

To clarify, consider a simplified comparison of how both Python and Fortran save binary data:

Python saves immediate byte values.

Fortran wraps the data with a header that includes the size of the block being written.

Running Python's tofile() does not generate these headers, hence why reading it directly as an unformatted file could lead to errors.

The Solution: Modifying the Fortran Code

After troubleshooting, you can correct the initial approach with these updates:

Step 1: Open the File as a Stream

You need to modify your Fortran file opening statement to treat the file as a stream. Here’s how:

[[See Video to Reveal this Text or Code Snippet]]

Step 2: Read the Header if Necessary

Add this additional step at the beginning of your read process, assuming your data might have headers or metadata:

[[See Video to Reveal this Text or Code Snippet]]

Final Working Code Example

Here’s how your complete Fortran program might look after modifications:

[[See Video to Reveal this Text or Code Snippet]]

Conclusion

By switching the file access to stream mode and correctly reading headers, you can successfully resolve the Fortran runtime error: I/O past end of record. Understanding the intricacies of how data is written and read between different programming languages is essential to troubleshoot such issues effectively.

With this approach, you can now read large binary files in Fortran confidently without encountering runtime errors. Happy coding!