Fixing corrupt shapefile?

Question

I have a shapefile that I was working on in ArcGIS Desktop 9.3 get corrupted somehow and it gives me the error "Number of shapes does not match the number of table records.". What I would want is to recover this shapefile as it has a lot of edits that other revisions of the file do not have.

Earlier I recovered from this error using ogr2gui but the tool now just crashes. I have even tried the command line version ogr2ogr but still no luck. This script, Shapefile Repair Tool from the ArcScripts website didn't help either.

Any ideas? (except that I stop using shapefiles)

Here is the output from ogr2ogr (based on a a suggestion in the answers): enter image description here

It almost surely became corrupted because its attribute table (a .dbf file) was being separately edited, creating an inconsistency between the shape records and attribute records. Even if you manage to "fix" the shapefile, make sure to do a very careful double-check that the correct attributes are associated with the shapes! — whuber, Mar 28 '11 at 17:29
The the error messages reveal much. Those numbers of points and parts indicate values between 2^30 and 2^32 (unsigned), strongly suggesting extensive physical overwriting of record headers in the .shp file itself (because they would not be valid counts, ever). It's therefore highly likely the data in those records has been overwritten, too. In short, your shape data probably no longer exist, at least in many portions of the .shp file, and therefore are not recoverable by software. Restore from your backups and start from there. — whuber, Mar 29 '11 at 17:30
@whuber. Thanks. Sure is the case. Each file recovered from this corrupt shapefile contains way fewer records than my backup. I've since reverted to the backup. — Erick, Mar 30 '11 at 06:44
Brad Nesoms recommendation shapecheck.exe, fixed my corrupted shape file first time! great tool — , Oct 16 '13 at 19:18

score 9 · Answer 1 · answered Mar 28 '11 at 08:36

9

An option, with the lost of the corrupted features, could be to use ogr2ogr converting your shapefile to another shapefile, with the -skipfailures option:

ogr2ogr -skipfailures fixed_shapefile.shp corrupted_shapefile.shp

for further details look at this blog post from PerryGeo blog:

http://www.perrygeo.net/wordpress/?p=132

answered Mar 28 '11 at 08:36

capooti

2,698
21
22

Ran the command (including -f "ESRI Shapefile" you left out). Got error report on the file.[Can't post screenshot here]. It has to do with the culprit shapes I guess. The format is: ERROR 1: Corrupted .shp file: Shape 2352, nPoints=7, nEntitySize=88. – Erick Mar 28 '11 at 09:11
you can safely remove the -f option: shapefile is default output format. It would be interesting, for further research, if you could attach a copy of the shapefile – capooti Mar 28 '11 at 09:28
File for research: I would have gladly forwarded the file but unfortunately the data is copyrighted. Thanks anyway. – Erick Mar 28 '11 at 10:13
@Erick, if you put the screenshot on http://imgur.com/ I can post it here (if still relevant) – djq Mar 29 '11 at 00:22

Brad Nesom · Answer 2 · 2017-09-23T01:20:59.283

5

I use this one. It has always fixed that problem for me.

shapecheck.exe

edited Sep 23 '17 at 01:20

answered Mar 28 '11 at 14:25

Brad Nesom

17,412
2
42
68

I Agree with Brad, ShapeCheck just works - standalone fixes shapefiles - truncates when required. – Mapperz Mar 28 '11 at 17:05
@Mapperz. Ran the tool, with several truncates. Could indeed open the shapefile thereafter! But it now contains fewer records than the backup I have to revert to. If I had no backup a recovery like this would have been it. (Archiving tool for eventuality)Thanks. – Erick Mar 29 '11 at 06:41
Link is broken. Here is working one: https://geonet.esri.com/servlet/JiveServlet/download/614216-1-160601/shapechk.zip Source: https://geonet.esri.com/thread/177037#comment-614216 – Comrade Che Sep 21 '17 at 10:15

score 4 · Answer 3 · edited Jul 07 '14 at 14:49

4

The Official Answer from Esri has a number of tips but points you to the shp repair utility which has saved me getting fired a few times.

Few other ones that I have heard of but can't say I have tried:

edited Jul 07 '14 at 14:49

Brad Koch

475
5
23

answered Mar 28 '11 at 07:46

jakc

9,858
8
49
97

Thanks Simon. But my major hitch is I can't get the file to open for starters so most of the suggestions won't work. Have just tried everything on Tip One you suggested. Nothing worked. – Erick Mar 28 '11 at 08:00

score 2 · Answer 4 · answered Apr 13 '11 at 21:53

Ok, here's another trick to add to the pile of good answers above.

This one's a bit more brute force, most of the time it helps, sometimes it doesn't, and while it's probably only a first step toward fixing the problem (rather than a solution unto itself, which it's often not), it can help get you to where you can open the shapefile. Most cases you'll still need to do more manual repair in ArcMap after the shapefile opens (corrupt features?, missing attributes? misaligned attributes? etc.)

Copy the shapefile into a new empty folder. Only take the SHP, SHX, and DBF with you. Leave all other files behind, and yes that includes the prj.
(Windows): right-click the SHX file and select "Properties" to open the file properties.
On the "General" tab take a look at the EXACT file size of this SHX file down to the byte. Look at the "Size" property, and not the "Size on disk" property.
Take that file size in bytes and subtract 100 bytes (the header). Of the remainder, divide by 8 (the size of each "word"). The result gives you the number of shape features inside the SHP part of the shapefile.
Open the DBF in some software that is going to allow you to edit the DBF and save it back out as a DBF. Add or remove records in order to make the rows in the DBF match the number of shape features in the SHP you calculated in step #4. (If you're using an older version of Excel, keep in mind that Row #1 contains the field names, so if you're going for 1,000 records, that'll end up being 1,001 rows in the sheet since the first data row is Row #2.) If in order to make the number of rows match you needed to delete rows and those rows had real data in them you need to keep, just save those out to a new DBF, and you can readd that stuff back in later, once you've gotten to the point where everything opens up again in ArcMap.
Once you've used the steps above to match the number of features in the SHP with the number of rows in the DBF, try to open the shapefile again in ArcMap.

score 1 · Answer 5 · answered Mar 28 '11 at 08:14

1

You can try to count the number of shapes into your .shp files with ogrinfo (not sure it will work) :

 ogrinfo -sql 'select count(*) from myshp' myshp.shp

If you are able to count the number of shapes you could then open your .dbf file with open office to complete it (or removed extra rows)

answered Mar 28 '11 at 08:14

simo

8,580
1
28
56

Thanks for the suggestion, but I'm not exactly an GDAL tools pro. or 'very' sql literate. Tried running tool with parameters you provided but got tool related error. "Unable to open datasource count(*)". – Erick Mar 28 '11 at 08:39
could you copy/paste the command line? – simo Mar 28 '11 at 08:53
I can but still same error. But @capooti 's solution seems to have produced some results/ identified culprit shapes. – Erick Mar 28 '11 at 09:24

score 1 · Answer 6 · answered Mar 28 '11 at 14:13

1

If your shapefile was a point layer and had XY field values, you could run the Make XY Event Layer tool to make another layer from the corrupt shapefile dbf.

answered Mar 28 '11 at 14:13

artwork21

35,114
8
66
134

Its a roads (polylines) file. But will keep your hint in mind for a points scenario. Thanks. – Erick Mar 29 '11 at 06:44

score 1 · Answer 7 · answered Mar 28 '11 at 19:03

1

I've had some luck deleting the index files (.idx and .shx), which your GIS will regenerate when reconnected.

answered Mar 28 '11 at 19:03

Nate

398
2
7

Tried this without success. Thanks for suggesting. – Erick Mar 29 '11 at 06:42

Fixing corrupt shapefile?

7 Answers7

Linked

Related