We are creating a tar file using 7-Zip and then extracting the data using WinZip. For one xml file (20180814-open-index.xml), the extract does not match the original source. If we extract 20180814-open-index.xml using 7-Zip the files match. The differences consist of one added character and one dropped character, where both positions appear to be in very unique locations (500,000 and 700,000 hex character positions within the file). This is a very unusual problem for us and we are not able to decipher if this is a problem associated with the data, our environment, the COTS WinZip product, or the 7-Zip COTS product. Any insights are greatly appreciated.
winzip tar file smart cr lf conversion
Compare results and our 7Zip license and system information are attached. Also attached is the xml file 20180814-open-index.xml file that is the data source for this problem. You can duplicate the problem by using 7-Zip to create the tar file, extract using WinZip, and compare to the original file. You can also extract using 7-ZIP and verify that it does match the original file. If for some reason that you are not able to duplicate the problem, it may indicate that something in our environment is the cause.
Each of these files, in turn, can be uncompressed into two separate files,a text header file with a .txt suffix, and a binary rasterfile containing "big-endian" values with a .dat suffix.
Name this file the same as the .bil file, except with a .hdr file suffix.(In the example above, the new header file name would be "us_ssmv01025SlL00T0024TTNATS2006010105DP001_new.hdr".)
Open ArcMap and choose File -> Add Data. Locate the location of the new .bil (and the not shown .hdr) file, and click on the file to add the data. You will see a range of values for the raster. The value -9999 is the "no data value". You may also see 55537 instead, which is due to Arc treating the file as "unsigned" (mapping negative values to high positive values) rather than properly positive/negative signed.
It is complaining that the SHA1 hash of the .ovf and .vmdk files does not match the actual file. I re-downloaded the tar.gz file multiple times and the MD5 hash verifies ok. The archive unzips fine. Tried multiple versions, 5.4.1, even 6.0.0 and they all come back with the above error so I cannot even deploy the appliance in vSphere. Is it possible that Cisco has posted VM images that contain incorrect manifest files? Or perhaps manifest files that are signed by a certificate that is not trusted by most computers out there? I really need to get this up and running quick...
I was using Winzip and although it doesn't break the MD5 of the tar.gz it affects the SHA1 hashes. Solution was to delete the .mf files before importing via vCenter. That way vCenter doesn't produce an error. Thank you for helping.
VMap0 is divided into four libraries based on geographic areas. The above map shows these coverage areas. Each library is on one CD-ROM or Zip-file as follows: Disk 1 North America (noamer), Disc 2 Europe and North Asia (eurnasia), Disc 3 South America, Africa, and Antarctica (soamafr), and Disc 4 South Asia and Australia (sasaus). These files were available from the main NIMA website, but they have now been removed. As of 2020, you can still download the VMAP0 data from the Indiana University Virtual Library, as '.iso' files.
Installing VMAP0Once you have downloaded one or more of the files, the next step is to install them on your computer. This is quite a simple procedure if the following steps are followed carefully. The only requirement is a copy of WinZip or a similar program, to un-zip the files. The installation process is completely non-invasive. All it does is to move files to the new location. No changes are made to the Registry or any other System Settings. The file sizes when un-zipped are:
The first and most important thing to do is to check your WinZip configuration. Failure to do this will result in the files being corrupted with added bytes. Start WinZip, and go to Options>Configuration>Miscellaneous. Check that the white box next to 'TAR File Smart CR/LF conversion' does NOT have a tick in it. If it does, click to remove it, and click 'OK'.
The installation procedure can now begin. Create a new folder anywhere on your computer. Start the Winzip program, click on 'Open', and browse to the downloaded file(s). Click 'OK' to select it, and the contents of the Zipfile will be shown - this will be a surprisingly large number of files - 6877 for South America/Africa ! Click 'Extract', and enter the name, or browse to, your newly created folder in the 'Extract to' box. Select the 'All Files' and 'Use folder names' options. (The folder-names option is absolutely required for other programs to be able to make sense of the data.) Click 'Extract', and WinZip does the rest for you. If you have multiple files to un-zip, repeat the above procedure, keeping the same folder-name in the 'Extract to' box for each one. Close the Winzip program.
Visit the NGA site at and drill down to their geodata downloads page (currently at ). NIMA may choose to change their site at any time so this topic does not provide a detailed description of how to locate and download data from NIMA's site. If for any reason NGA no longer provides VMAP, users should file a Freedom of Information Act request to obtain this data.
For this example we will zoom into Central America and download VMAP Level 1 as a Mil-spec, G-zipped file. The result will be the download of an approximately 100-megabyte file called v1155.tar.gz. At the present time NIMA is providing its VMAP1 downloads as large files, each of which corresponds to the compressed content of one of the VMAP1 CDs. The v1155.tar.gz file corresponds to VMAP 1 ("v1") CD 155. There are over 200 CDs in the complete VMAP1 data set.
A .gz file is a "gnu-zipped" file that uses open source software to perform compression similar to Windows .zip files. .tar files are UNIX files that are created by the TAR ("tape archiver") program, which performs a function in UNIX similar to that of the arj program in DOS and UNIX. TAR can take many files organized in directories and combine them into a single file. It's a handy way of bundling related files together for transmission. .tar files must extracted into the many smaller files that comprise them.
When we download v1155.tar.gz from NIMA's site, what we've downloaded is a .tar file (a collection containing numerous files inside the .tar file) that has been further compressed into a .gz file. We can decompress the .gz and also extract the .tar file it contains using WinZip.
To do this, place the v1155.tar.gz file into a folder and double click it [assuming, of course, you've already installed WinZip on your system]. Note: before using WinZip to decompress files of this type, launch WinZip and make sure that WinZip's Option - Configuration - Miscellaneous tab does not have a check for "TAR file smart CR/LF conversion."
WinZip will open the .gz and see the compressed .tar file it contains. It asks us if we want to decompress it and open the .tar file. Click Yes. The following decompression process may take a while on a slow machine. WinZip will then display all of the files in the .tar file. Extract them all to a convenient directory, preferably named for the originating file. For example, we would extract all of the files from v1155.tar.gz into a directory called v1155.
Launch Manifold. Before importing, create a folder in the project pane named v1155. Importing VMAP results in a huge number of files, so it is best to keep them organized within their own folder in the project pane.
To import, choose File - Import - Drawing and choose VMAP Files (dht) in the Files of Type box. Browse over to the folder where you placed the files extracted from v1155.tar.gz, browse down into the directory tree into the soamafr folder and double-click on the dht file that is located there.
The dht file is a catalog of what is available within the numerous files in various subdirectories below it. The Import VMap Data Set dialog allows us to choose which data we wish to import. For this example, we will choose Hydrography and Transportation and press OK.
The 1155 file covers a very large area because it includes a data set that is normally provided on the VMAP1 155 CD. The illustration above shows the hydro map zoomed in to central Panama with all layers but the Hydrotxt_lib_155_hydro layer turned on.
A great site to get information to brief or just get general SA is the Navy'stropical weather Shapefile site at -bin/listing.pl?cen+shapefiles+tropical+Tropical_Shape_Files. Sure you can look at a generalized map on a website, but wouldn't you reallyrather see the path of the hurricane on the maps you already know? Thesite includes worldwide tropical Shapefiles so it isn't just for those of use inthe USA.
If you're looking for good vector data to add to your map and you're in theUSA you can download the "raw" Shapefiles that are used in theNational Atlas from Civilian authorities are concerned with things like countyboundaries, roadsand citylocations. If you're using WinZip be sure to turn off "TAR filesmart CR/LF conversion". With many/most files this causes the filesto be unusable.
The Commercial Satellite Imagery Library (CSIL) remains operational for its customer base.Using WARP you can download current, commercial satellite imagery withresolutions of 1 Meter or better directly to your PC. You'll need anaccount and many of these files are very big. You'd best judge youravailable bandwidth and only try to download image "chips" (asubsection of an image) if you find that you're bandwidth impaired. On theplus side, if you've got an account you can also log on at home and downloadovernight - many find they have better bandwidth at their house than atwork. Up until a few weeks ago you would have been limited to downloadingimagery in the NITF format (works with FalconView 4.x, but not any fieldedversions) but fortunately NGA has implemented a converter to translate fromNITF to GeoTiff. That means you can use the imagery inside FalconViewtoday. On the unfortunate side, the GeoTiff converter is only implementedas a temporary feature and is at risk of being pulled after 90 days. 2ff7e9595c
Comments