Shapefile Questions...

[aksnowbiker]

I've downloaded the "parcels" shapefile for the Fairbanks, Alaska borough (county to most folks).  I don't have much of a frame of reference, but it seems like a big file, about 22MB for the .SHP file.  The data is WGS 84.

I'm running TF Pro 3.12 on Win XP SP2 on a 1.4GHz AMD Athlon, 512MB RAM.  

With this shapefile loaded, overlaid on a 16M topo,  TopoFusion uses over 300MB of RAM.  The system is swapping madly at this point, and is acting really sluggish.

TF appears to load the whole thing, regardless of what extents are being displayed.  In any case, it seems like a lot of memory usage for what I think is a vector file.  Should I be approaching this in a different way?  

Ideally, I'd like to see the shapefile be a true vector overlay, which doesn't take much memory to display, but perhaps that's an idea for the "Feature Suggestions" area.

I suppose that I could save the shapefile as a .GPX file and devise a means of chopping it into managable chunks, except that the resulting .GPX file is almost 100MB, and fairly unmanagable in its own right...

TF seems to try to display the whole mess as a single track, or as a singular collection of tracks.  I suppose I could try making a JPEG of the thing and apply it as a user calibrated map, but I'd prefer to retain the data in vector format if possible, without the performance hit.

Finally, is there a preferred datum to import to TF?  My first attempt used NAD 27 data, supposedly in Zone 3, but that put the borough about a thousand miles off the coast of California.  Ironically, that's about where Alaska and Hawaii usually show up on national weather maps...  

If all this seems like I'm ragging on TF, let me say that it is a fine product, worth more than what you charge for it!

Thanks,
Tom

[Krein]

That is a rather large file (100 mb GPX file), but the main suggestion is to upgrade to v3.18.   You will notice a massive increase in speed, esp. when zoomed in on areas such that not all of the parcels are being displayed.

There was a problem in 3.12 and 3.15 that messed up the clipping code for tracks.

That memory usage does seem a little extreme.  I will take a look into it.

As far as input datum, the preferred format for all shapefiles is lat/lon.  Shape is such an archaic format that it doesn't store utm zone or projection.  WGS 84 (or NAD 84) and lat/lon make life much easier.

That said, there's an input dialog that hasn't been written in TF that should ask for the UTM zone.  Currently I believe it simply assumes the zone the map is currently centered on.  

Thanks for writing in, and give 3.18 a try and see what you think.

.

[aksnowbiker]

My mistake, I'm running TF 3.18.  I don't even know where 3.12 came from.  Fertile imagination, I guess.

[Krein]

Ah, OK.

One question, regarding memory usage - are you using DXT1 compression (Options->Prefs->General) ?

Do you have a link to the shapefile so I can download and check it out?

Thanks.

[aksnowbiker]

Yes, DXT1 compression is turned on.  A friend of mine (who introduced me to TF) tried the same shapefile on his three computers.  One 512M, one with 1G and one with 2G.  The 512 machine choked horribly, the 1G did better and the 2Gig machine operated just as fast as with a small file.

So, I guess that there are just too many nodes/vectors/whatever for my machine.  It is covering a few hundered square miles, after all.  

Here's a link to the shapefile in question in the event that you want to look at it:

ftp://co.fairbanks.ak.us/GIS/parcels_WGS84.zip  (22M zip)

[Krein]

Interesting.  It's not the number of points but the number of individual tracks that is causing the memory bloat.

I did some tightening of the structures and got it from ~400mb to ~140 mb, on my machine (with that file loaded).  We had some more or less unused structures in there for some of the more esoteric features of TF.

Thanks for posting this.  I was confused because I'd been working on a file of similar "size" but didn't see such a memory hog.

I'm guessing you've registered, and if so I can put up a beta version with the mods very soon.

I'm still working to get files like this to display faster.  Currently if you view the whole file from far away it's a little sluggish, but when only looking at part of the file, it runs well.