Re: Importing FITS files into Mathematica

*To*: mathgroup at smc.vnet.net*Subject*: [mg121166] Re: Importing FITS files into Mathematica*From*: David Bailey <dave at removedbailey.co.uk>*Date*: Thu, 1 Sep 2011 06:04:44 -0400 (EDT)*Delivered-to*: l-mathgroup@mail-archive0.wolfram.com*References*: <j3h9a8$ni1$1@smc.vnet.net>

On 30/08/2011 01:02, marko at astro.cornell.edu wrote: > Hello, I am an Astronomer, and am impressed with all the new features > in Mathematica 8. I'd like to be able to migrate some of my work away > from IDL. So I used a colleague's copy of Mathematica for a test run > before buying. I found what seems to be a serious deficiency for my > work. > > 3-dimensional FITS files are the bread and butter for radio > astronomers. Mathematica seems to be able to handle FITS with 2-D binary > tables no problem. When I try importing a 2-D binary table with > Import["filename"] everything works fine. However, if I use a 3-D binary > table with Import["filename"] I get the following message: > > ToExpression::sntxi: Incomplete expression; more input is needed . > > Doing Import["filename", "Elements"] works just fine, yielding: > > {"Airmass", "Author", "BitDepth", "ColorSpace", "Comments", "Data", \ > "DataType", "Declination", "Device", "Equinox", "ExposureTime", \ > "Graphics", "History", "HourAngle", "Image", "ImageSize", "Metadata", \ > "Object", "ObservationDate", "Observer", "Organization", "Plaintext", \ > "Range", "RawData", "Reference", "RightAscension", "SiderealTime", \ > "TableData", "TableHeaders", "TableUnits", "Telescope"} > > But trying to extract any individual element, such as > Import["filename","Author"] again yields: > > ToExpression::sntxi: Incomplete expression; more input is needed . > > > I tried this with several 2-D and 3-D fits files (all of which I know > are good and in proper FITS format) and it seems to work just fine with > 2-D. I'm pretty sure I've got the syntax right since the only difference > between working and non-working results is a 2-D filename or a 3-D > filename. > > This is a necessary functionality for me. Am I doing something wrong or > is Mathematica actually unable to read FITS files with 3 dimensions? Is > there a package or an add-on that I'd need to get? I was able to find an > implementation of WCStools for Mathematica online but that also only > seems to handle 2-D fits files. > > Would it be possible for me to build a function to read these cubes? > Would this be a difficult task or would it be just a few lines of code? > Keep in mind I've only been playing around with Mathematica for one day. > > FITS files have a relatively simple format. An Ascii header which > typically has about 80 lines, but always ends with an "END" line. This > is then followed by binary data as either BYTES, FLOATS, or DOUBLES. > > Thank you for any advice, > Marko Krco > > Import is a high level function which obviously works on just those file structures which it recognises. There are always ways to read data into Mathematica - since it can read data in a variety of formats, including binary data. Do you really mean that after a text header, the rest of the file is in binary? If so You should open the file with OpenRead using the option BinaryFormat->True and then use BinaryRead to read the data. It may be worth considering how large these files are. If they are larger than (say) 200 MB, you may want to consider some or all of: 1) Reading in only part of the file at any one time. 2) Ensuring that you end up with Packed arrays, which use less storage, and are also more efficient. 3) Installing Mathematica on a 64-bit OS - which will automatically install the 64-bit version, and give you access to a much larger address space. Alternatively, if you already have a program - say in C - that can read these files, it may be easier to interface this to Mathematica. In 8.0 you can make direct calls to C functions in a DLL. I hope this helps, David Bailey http://www.dbaileyconsultancy.co.uk