v3.83 released

[ScottMorris]

v3.83 was released today.  It fixes some issues downloading color aerials, and also adds Maine to the list of covered states.

Details:

3.83 – [8/15/09]  (Beta)

Fixed Color Aerial downloads for the east coast, due to server change
Added Maine to Color Aerial tileset

[ScottMorris]

Also, 3.84, a quick fix of a somewhat rare PhotoFusion / GPX issue.  Basically only affecting photo paths with ampersands in them.  But also a little robustness on the GPX reading code.

3.84 - [8/16/09]  (Beta)

Fixed GPX issue with PhotoFusion paths containing ampersand
Improved robustness of GPX reading

[ScottMorris]

More was added to v3.84, and then today v3.85 with another fix:

3.85 - [8/18/09]                                                                                                 
                                                                                                                 
Fixed issues with Color Aerial server(s) and Persistent Connections enabled                                       
Re-enabled "TopoFusion is having trouble contacting" message                                                     
   (inadvertently disabled in previous betas)                                                                     
                                                                                                                 
3.84 - [8/16/09]  (Beta)                                                                                         
                                                                                                                 
Fixed GPX issue with PhotoFusion paths containing ampersand                                                       
Improved robustness of GPX reading, including CDATA in args                                                       
Cleaned up PhotoFusion memory management                                                                         
Fixed Map menu tileset shortcuts not working when focus was not on map                                           
Added shortcut to 'blending options' (image processing) from map menu                                             
Disabled non-functional menu name in draw/merge track context menu

-----------

I think this version is good to go.  Will release as full public release very soon.

[sanewcomb]

Hi Scott,

I'm using 3.85, but this has been a problem for some time going back 6 months or more, and it's getting worse. TF cannot work for more than 10-30 seconds without triggering the tile downloading toggle, shutting it off. At first I thought it was just a slow server, particularly the color tile servers, but it now happens on a regular basis with Terraserver. It's not that much of a bother with Terraserver yet, though annoying since it hasn't behaved like this ever in many years of using TF. But on the AZ color server it makes it practically useless. It shuts it off after loading 8 or 9 tiles, and it takes some time just to get those tiles.

I thought the batch download might help, but the behavior is just mimicked because I guess the batch procedure just runs TF through its paces. I can see the tile downloading toggle go on and off as it runs. So if I wanted to get a complete map record of the Catalina Mts it looks like it will take many days or weeks for just this small area. Is this how it's suppose to work? Are the servers set to just spit out 8 tiles at a time? Or is there something going on in my setup that is triggering this behavior? It seems like others are getting better results, although I don't know how many are using the color tiles or batches. I have not been using the batch tool at all for several years and only started using it today to check out the upgrades in 3.85

I do enjoy the color tile set for AZ, but when I want to look at an area, or research a few areas for an upcoming hike for the weekend, the time and effort waiting for the tiles to come across is very frustrating. What should take 10-20 minutes ends up taking several hours due to the waiting. And I really can't use the download tool to speed it up because it takes much longer to download a square block of all the tiles then move the map around in real time reacting to what you see in the aerial images.

Thanks for any help,

Steve

P.S. Do the AZ color servers specify limits or speeds for serving up tiles? Could a few TF users be using up all the quota within a time period, making them work at a reduced level for all other users (TF or otherwise)?

[ScottMorris]

Hey Steve,

I have noticed the issue with the AZ server lately as well.  Haven't had time to look into it yet, but I will soon and will post what I find.

Terraserver, on the other hand, has been working flawlessly for me.  That's troubling.  Can you post or email what your TF internet settings are?  (i.e. Preferences->Internet).

Thanks.

[ScottMorris]

It seems the server just periodically gives no response (zero bytes).  Not sure why, or if TF users have anything to do with it.

But, I think something changed recently with respect to when TF throws a "can't contact" error and turns off downloading.  I've made it more tolerant of bad / no response.

You are right that batch downloading just drives TF downloads, so there won't be much difference except the automatic re-trying.  I'm downloading about 100 mb of color tiles from AZ right now and haven't yet seen any disabling of downloading.

If it is successful, I'll put out a beta soon.  May have to disable the shaded relief stuff, I guess.

Thanks Steve.

[sanewcomb]

Can you post or email what your TF internet settings are?  (i.e. Preferences->Internet).

No Proxy Server
25 Connections
Restart Connections after 20 Sec (though download dialog always shows 60 s)
Use Persistent Connections checked

I've noticed a few things about the download function. Although it claims it has downloaded 100%, there are missing tiles in Topo, Aerial and color (many) tilesets. It's probably pretty difficult to figure out the actual tiles brought down given TF stops after some time if the server doesn't send it. Redoing the download over the same area seems to work well picking up the missing tiles and doesn't repeat the downloads. Would be nice to be able to upload the GPX box to repeat downloads for missing files. Also a pause function would be nice, but neither high priority.

I just started a 3rd round, after a 2nd round to pick up missing tiles went pretty fast and only adding the Tiger set. The Tiger tiles should be small, but it triggered a lot of retries and is taking a lot longer than I would think. Could just be a transient Tiger server hickup.

It finished. It reports 100%, 0 tiles not present on server, but after 2 iterations there are still many missing Tiger tiles. It's probably just that server right now, but the stats reported by the dialog box aren't very useful. Is there any way that TF can actually see if there are missing tiles? It is a royal pain to manually scroll across the screen, each time for each tile set, to see if all the tiles are actually there.

Hopefully there is some sort of problem with my setup. This download toggling and missing tiles are affecting the usability of the program for me. Mostly, though, just using it in real time to scan areas.

Thanks,

Steve

[sanewcomb]

I've noticed you can just hit the Start Download button over and over again without redrawing the box. I guess that's a work around. Keep doing that until it reports 100% and 0 tiles downloaded during that attempt.

[ScottMorris]

Steve,

Yep, others have reported that it will finish and still have missing tiles, even with a second pass.  Generally this has only been with people who are having connection issues, and most of this was fixed in 3.85.

I plan to make it continue making passes until everything is definitely downloaded in a future version.

I just put out v3.86 (beta) with what fixed AZ color aerial downloads for me (AZ is its own server).  Please let me know if it works any better for you.

It's very possible you were getting connection errors from queue'd up color aerials, even when you were in topo/aerial (Terraserver) display mode.  So, hopefully this fixes it for you.

Thanks.

[sanewcomb]

Giving 3.86b a try now. BTW, the Color and AZ tile sets seem identical to me. For the Downloading dialog box, would it make more sense to include one or the other, but not both? Seems users not knowing better will just download both, twice the tiles needed.

Steve

[ScottMorris]

Sorry, didn't mean to confuse the issue.  The AZ server is a part of the Color tileset - which is made up of multiple servers.  Most of the servers host multiple states, but AZ is its own, which can account for different behavior, that's all.

Any better/worse/same?

[sanewcomb]

Using the Color tileset around Tucson the real time panning seems to work better. I can pan around more without triggering the error and turning off the tile download.

But, the download function seems pretty much the same. The retry is triggered pretty quickly, and it only grabs 800 to 6000 tiles at a time before the download function ends and shows 100% downloaded, 0 left on server. This takes about 1 hr. At that rate it would take at least 10 hrs to grab the color tiles for the Catalina Mts and 10 clicks of the Start Download button, probably more.

There is one difference. Now after a couple iterations of Start Download it reports only a few tiles (18) to get and almost instantaneously stops, reporting 100%, 0 downloaded, 0 not present on server. BUT, there are still plenty of tiles missing. Previous routine worked better in that it would continue to get the missing tiles until all were retrieved even though the stats were reported wrong. Thanks for looking into this. Steve

[ScottMorris]

Steve,

Well, at least there's some improvement.  Nothing changed in the batch download tool code as far as tiles downloaded, already present, etc.  So I can't explain the difference you noted there.

I'll keep testing, but so far I have yet to see it complete a download and have missing tiles.  I also am not getting any 'downloading disabled' messages during the batch download.  Granted I have only done moderate sized batches. 

You might try decreasing the number of download threads.  The most recent batch I did was with 8 threads and it didn't seem to be any slower than 25.  For terraserver more definitely helps, but perhaps not for AZ color aerials.

[Larry]

"No Proxy Server
25 Connections
Restart Connections after 20 Sec (though download dialog always shows 60 s)
Use Persistent Connections checked"

Steve and Scott, for what it's worth, I had a similiar issue some time ago that I fixed by not using persistent connections. It's as if the server is saying 'you can't tie up 25 phone lines to me and expect me to send you anything'. At least that's the way I imagined it.

Larry

[ScottMorris]

Larry, thanks for your experience.  As far as I know Terraserver is OK with many persistent connections, but some of the color aerial servers don't like them.  In one of the recent betas I simply disabled persistent connections (even if it's checked) for all color aerial servers, and that seems to have cleared up a bunch of issues for people.

Steve -- there is a new beta version out with much improved batch downloading.  Missing tiles should be a thing of the past, and hopefully less 'server error' messages, faster downloads, etc.  Let me know if you have a chance to test it.