SketchUp plugin for ArcGIS 9.2 released

At the ESRI User Conference I had an opportunity to chat with folks from SketchUp team. My first question was "When are we going to see plugin for ArcGIS 9.2?". They were really hoping to get it done before the Conference but it didn't happen. The assured me it will be the week after the conference and the plugin is now available for download from SketchUp's download site.

If you haven't looked at SketchUp have a go with free version available from Google's site.

Heavy Metal cartogram

If you like Heavy Metal like I do and love maps, head over to CartoTalk and check "The First Heavy Metal Cartogram". It is interesting to see where in the world is the highest concentration of heavy metal bands and on the serious side, read comments other members have on this map.
If you have not been over to the CartoTalk I would recommend you do. One can easily spend hours reading threads and post on variety of topics.

75 MHz vs 2.66 GHz

This may seem as dumb question but people from Maryland Institute of Technology don't think so. Professor Uzi Vishkin and his team have developed a prototype of desktop computer using 75 MHz processors, 64 of them and it is big as a license plate.

The key is in the parallel processing. This idea is not new and Vishkin has been working on this since 1979. As he points out in the press release hardware is not the main problem, it is software that makes it hard. So they have written their own software to take the full advantage of this system.

Why this may be a challenge for other CPU and computer manufacturers? It shows you don't have to have 4 GHz or 1 THz CPU to get the job done very quickly. It is never a single side solution, like increase in CPU clock. Without software any computer is good for junkyard or shooting practice.

IBM - The king of supercomputing

If you have ever wondered which computer is the fastest then check the Top 500 web site. Some may be surprised to see IBM on the top of the list with BlueGene/L system. Reading about the configuration of the this beast makes computers I use like wind-up toys.

For example, "1,024 gigabit-per-second links to a global parallel file system to support fast input/output to disk" while I am tinkering with RAID 5 ...

On the rest of the list you can find usual suspects like Cray, SGI, SUN and other.

It is nice to see that there are systems from this list are located in New Zealand - University of Canterbury that is pumping out 2.8 GFlops.

It would be nice to have ArcGIS running on that...

ESRI UC - technical update

Originally I didn't plan to post such detailed summary of presentations I went to but friend of mine, Jithen Singh, wants to put it on his blog, Scrappad. Then I figured why not. Just a few notes about it; these are the details I think are worth mentioning, from my own perspective. I have been in the GIS since late 80's and I do a lot of training courses. Because of this you may find some sessions to be quite brief.

Anyway, here is the more to the point technical summary of the sessions I have attended.

VISUALIZATION WITH ARCGLOBE

• Use ArcGIS Online to download template documents. These may have higher resolution imagery available than default ones, at least for USA.
• There is a façade tool from ArcScripts (id 14040) that improved 3d building display
• Use layer properties to have multiple displays of 3D data (no texture, low resolution texture, hi resolution texture)
• www.mayang.com/textures is free resource for download of various textures
• ‘3D graphics’ is a new toolbar at 9.3
• KML will have separate toolbar at 9.3
• ‘Find nearby’ tool will utilize ArcGIS web services
• Use ‘Cache deployment wizard’ to manage and deploy cache

LABELING, ANNOTATION EDITING AND TEXT PLACEMENT

• Straight option for polygon labeling is using longest axis of the polygon

ARCSDE CONFIGURATION AND UPGRADE STRATEGY

• We were given handout sheet with steps for upgrade (client and server, PC and Unix)
• Unicode support is enabled by default in 9.2. If not needed turn it off
• High precision allows storage of 4 503 599 627 370 000 geodatabase units (exact number)
• If using application server connection upgrade clients first
• If using direct connect upgrade simultaneously
• KB 31494 discusses support for 32 and 64 bit ArcSDE

MOBILE GIS USER GROUP MEETING

• Windows Mobile 6 comes in 3 flavors:
  • Standard (no support for touch screen, 2 tier security)
  • Professional (support for touch screen, 3 tier security, includes Remote Desktop Mobile client)
  • Classic (for PDA, non smart phone, etc)
• SILVERLIGHT RUNS ON LINUX

ADVANCED MOBILE BUILDER

• Calculate value tool is designed for model builder, can use code block
• 'Merge Branch' tool should be used in combination with precondition property to complete if/then
• Use !shape.area! statement in expression(s) to get access to area of features in a shapefile without adding and populating new field
• Probably there was more but I got sick and had to leave…

ARCGIS SERVER ADMINISTRATION, CONFIGURATION, TUNING AND OPTIMIZATION

• ESRI’s recommended connection type is direct connect (open to discussion IMHO)i
• For dynamic map services:
  • Use minimum number of required layers
  • Hide unnecessary fields
  • Use simple or character marker symbols
  • Use EMF instead of BMP for symbol fill
  • Use integer fields for symbol values instead of character values
  • Use ESRI_Optimized style (>50% speed increase)
  • Use annotation instead of labels
  • Build indices on fields
  • Use label weight and label priority sparingly
  • Avoid special effects on text (halo, background, etc)
  • Avoid text with 60+ size
  • Avoid Maplex
• For cached services set number to number of CPUs or Cores + 1
• Simplify data (generalize) and models
• Us in memory data for geoprocessing (models)
• If you have more than 4 threads per SOC process use high isolation
• For polled instances use 2-4 instances per CPU or core
• Citrix NetScaler can boost up network performance
• Have at least 2 GB of RAM per CPU or core
• Reduce number of services per server
• Use NAS, RAID and fast SCSI disks
• Use Server Statistics in ArcCatalog for investigation of possible issues
• Use Microsoft’s Fiddler for investigation of HTTP communication (www.fiddlertool.com)
• On XP disable Simple File Sharing
• Use web stress tools like MS application Test Centre or Jakarta’s Jmeter

MANAGING DISTRIBUTED DATA WITH GEODATABASE REPLICATION

• Replication can be used with other data distribution techniques like Mobile ADF
• You can replicate specific version, dataset of features
• 3 replication types are
  • Check In / Check Out – parent must be ArcSDE
  • One way – can be simple of full (support for networks and topologies)
  • Two way
• YOU MUST HAVE WRITE ACCESS TO DATA
• All data must be registered as versioned
• All data must have GLOBALID column
• All data must be in high precision
• For terrains, only components are replicated, not the terrain itself
• Metadata is replicated this is in the works for 9.3
• Connected synchronization – both GDBs are on the network, can send changes both ways or one way
• Disconnected synchronization – messaging done by user, gdbs are not on the network
• Send acknowledgment messages to reduce amount of information sent, also helps cleanup of data
• Replica reconcile policies
  • Favor database
  • Favor import – apply imported value
  • Manual – user must resolve conflicts
• Use Synchronize Changes GP tool for automation
• For schema changes
  • Field dropped – skips the field
  • Apply schema before synchronization
• There is new feature class, replica footprint created by geoprocessing tool. It includes replica properties
• Changes in 9.3
  • Extending replicas to file and personal gdb
  • Support for XML files over 1 GB
  • Improved logging in case of unsuccessful synchronization

ADVANCED CARTOGRAPHY IN ARCGIS

• Check samples in developer kit to create your own geometric effects
• New at 9.3
  • Greater selection handles
  • More keyboard shortcuts
  • Dialogs to convert units of measure
  • WYSISWG for editing representations
  • Disperse markers symbols GP tool

ENTERPRISE GEODATABASE TUNING TIPS AND TRICKS

• Client’s responsibility
  • Better workflow design
  • Better geodatabase design
  • Optimized ArcObjects code design
  • Optimized MXD
  • Testing
  • Understanding the technology
• Remote SDE install - for unsupported database
• Remote SDE install, CPU clock is the key, RAM is secondary
• Check PERMISSION_CACHE_TRASHHOLD keyword for SQL server at 9.2
• Use SP2 for SQL Server 2005

• Use built-in database roles like db_datareader (reduces time for reading of system tables)
• Sde commands use single precision
• User can control logging mode - ERRLOGMODE keyword
  • T- time stamp
  • C – computer
• Read KB 25129 about TCPKEEPALIVE setting

On personal note, it was a great opportunity to meet ESRI staff and other people whose books or papers I recommend to my students. This includes Cynthia Brewer, Peter Jolly and other.

Samsung leads the way

Again, Samsung shows leadership when it comes to SSD (Solid State Disk). They have started mass production of the 1.8 " 64 GB SSD drive. It is based on the single level cell flash memory. On the practical side this means that battery life of your laptop could be extender by 20%.

On the other front they have released hybrid disk drives, MH80. These drives are combination of standard disk drive (80 or 160 GB) and 256 MB flash memory. This was only a matter of time before someone does it. Again, on the practical side this can significantly reduce power consumption of laptops and faster resume from hibernation.

It still may be quite some time before we see laptops with SSD drives only. Reason is not technical this time - it is the price. Over the time prices will drop but until then keep your eyes focused on Samsung.

ESRI UC, Tuesday and Wednesday

The conference is in the full swing. From early morning till evening there are technical workshops, paper presentations, special interest group meetings and variety of other activities.

The workshops are done very professionally. Presenters are excellent and quite often work in pairs. Topics for presentation rage from "What is GIS and how it can help you" to "ArcSDE configuration and upgrade strategy" so everyone can find something interesting.

The biggest problem is to decide what presentation(s) to see. For every time slot I had 3 or 4 options. And even with two of us there were still uncovered tracks. If you want to see the presentations that you missed there is an option. All technical workshops are recorded and synchronized with power point presentations. You can order a set of DVDs from TwoSense web site. If you order before 22nd of June it will cost you US$499.

I don't want to retell the presentations here but to point some of interesting details.

In ArcGlobe 9.3 some of new features are:

• billboad picture symbols
• KML 2.1 support
• terrain supports interpolate shape and line of sight geoprocessing tools
• profile graph supports multiple lines in one graph and can also graph line of sight results

There is lot more I could add but it is getting late...

P.S. Be careful where do you buy lunch...

ESRI User Conference, Day 1

The Conference is now in full swing. There is over 13 000 people coming from 113 countries. That’s a big crowd. The conference centre is superb venue with enough room to accommodate everyone. And everything works. Technical setup of the stage is nothing less than impressive, 3 huge wide screens and excellent sound system.

As usual, it begins with plenary session. First speaker is Jack Dangermond himself. Presentation is very neat with feature rich slides in the style of latest Microsoft Expression applications – reflections, animations etc.

Back to the Jack. If you’ve been at the Conference before you know Jack’s passion for GIS and this year’s opening is no exception. Here are some of the points from his speech.

The theme of this conference is geographic approach. Strong emphasis is on us – users of the GIS and what are we doing with it.

Jack has presented Enterprise Application Award to Saudi Aramco, world’s largest organization. This was followed by President’s Award that was given to The Nature Conservancy.

In his address Jack stated that we “need to fundamentally change”, the way we work, make decision and act. He also gave us his definition of GIS as “a framework for understanding and managing our Earth”. This definition clearly shows his idea of GIS as the basic technology that will link other systems.

Over few last years we have witnessed emergence of the web as platform for implementation of GIS. Now we have GeoWeb with consumer visualization segment (Google Earth, Virtual Earth from Microsoft) and other (ESRI) with distribution of rich content like data, models, services, etc. as integrating process of GIS and everything in web.

We also got the taste or ArcGIS 9.3. It will be delivered as series of service packs and focus is on improving quality and addition of new tools and features. For example in desktop there will be new option for labeling polygons, specialized labeling options for contours and rivers, new street numbering style. On the editing side representations will be modified in “WYSIWYG” environment. Some other new options are textured 3D building models, support for time series, addition of the progress bar to geoprocessing tools, display of schematic diagrams over map data, etc.

Clint Brown showed us Survey Analyst extension at work and how it can get CAD data in local circuit and create point, line and polygon feature classes with ease.

As expected ArcGIS Server was hot topic. At 9.3 release focus is on the improvements in documentation is improved (much better), scalability and performance. Of course, there are new options like addition of image services from Image Server, addition of role based security model and support for mashups via JavaScript.

At 9.3 some new databases are supported like Postgress SQL, Oracle Express and DB2 z/OS.

ArcGIS Explorer will support REST (standard for web development), SOAP XML, JavaScript, KML, V.E. and whole range of OGS services.

As a part of new development ESRI has demonstrated ArcGIS Online web site that provides GIS content and services for GIS users. You can choose to use map documents, ArcGIS Explorer documents or layer files. Also, ESRI is adding 2.5 m SpaceGlobe satellite imagery as a free data/service. If you prefer to have data locally you can buy an appliance that has all data ESRI is hosting, kind of the “World to go”.

There are some interesting things happening in the support area. Support staff in ESRI is part of the development process, not only guys that take calls and pass it to the team. Due to large volume of the support calls with 9.2 release ESRI is increasing support staff by 30% so we should not wait long before issues get resolved. The biggest changes are addition of diagnostic reporting (for 9.3, I guess it is similar to Miscorsoft’s error reporting) and providing access to ESRI’s internal knowledge base.

In the second part of the morning David McGuire presented new features in Desktop, Server and cool new inventions.

In Desktop new features include suspension of all labeling with single click, introduction of zones for labeling of polygons similar to points, disperse marker symbol tool. The later one will be very useful for very dense (urban) areas and alike.

Next demonstration was JTX or Job Tracking extension. Standalone application GUI has a striking resemblance to Outlook 2003. New in this release is introduction of the job workflows that are represented as diagrams. In side window user can see step descriptions for any step in the diagram.

For analytically orientated users there is a great news. In 9.3 ESRI is introducing Geographically Weighted Regression tool. This tool includes spatial component in weighted regression model. There is also a new type of graph – Matrix scatter graph where user can choose one from pairs of variables from predefined set of graphs.

In 9.3 new functionality is dynamic display and it was demonstrated quite effectively in an ArcEngine application.

For the end, David showed some really cool innovations. First one (and most attractive one) was “digital pen”. Using this pen one can write on the paper map of any size, or multiple maps and his sketches and notes are transferred into the geodatabase by docking the pen to a computer. The trick is in the watermarks that are created on paper map and digital pen is reading.

Next demo was MetaLens from National Geographic that is a mashup of digital media and GIS data.

Linux fans were delighted to see use of the 8 GB flash drive that will boot up RedHat Linux, start ArcEngine application written in Java as a single step operation. This is great example of simplification and streamlining of processing.

For all educators OmniGlobe is very nice tool. In essence it is projector that uses inner surface of the globe as the screen. It is controlled by computer and can be very powerful tool.

You can find lot of details (and very up to date) on User Conference blog.

BumpTop desktop, again

On TED conference, Anand Agarawala has demostrated his BumpTop Desktop. It is nice and fairly intuitive replacement desktop based on the simple concept - simulate a real desktop with documents and piles. What is different in his approach is that it is including physics.

Have a look for yourself and discover how different your next desktop could be.

Preconference survey Q and A on-line

Just a quick update on previous post about ESRI's pre-conference survey. Questions and Answers are now on-line. Read them here or if you prefer single page layout go over here.

Cartographic Relief Presentation - reissue

Just like ESRI has heard my wishes. For a quite some time I was trying to buy this Eduard Imhof's book but with zero success. Since the first time I have seen his maps and read chapters of this book I knew I have to have it.

Even before I knew of Imhof I liked his maps. How? In the days of my primary school we had an atlas published in 1971 featuring maps done by Imhof. For people from my part of the world, it was "Nas svijet. Zemljopisni atlas svijeta za VI i VII razred osnovne skole." This is an excellent atlas and it was one of my favourite school books.

Back to the book. Here are the details about it and how to order it. It costs only US$60 (for an English translation from 1982 the asking price was about $300 from bookstores that actually had it). I'll post more comments again here once I am finished reading it.

If you would like to find more about the Eduard Imhof and his work have a look at the following sites:

http://www.maps.ethz.ch/imhof4engl.html
http://www.reliefshading.com/index.html
http://en.wikipedia.org/wiki/Eduard_Imhof

ESRI User Conference survey

It is only one week until the ESRI User Conference. As a part of the Conference preparation ESRI did a survey of registered users. Results of the survey will be on-line at some stage.

Questions and answers are grouped into the categories (19 all together) and they cover a range of topics, from ESRI in general to specifics of the product range. Large number of answers talks about the next release, 9.3. There will be a lot of new features and improvements including support for Microsft Vista.

Mars - up close and personal

University of Arizona has a new web site - HIRISE. On this site you can view images of Mars taken by the Mars Reconnaissance Orbiter. Right now, there is over 1100 images available for viewing and download.

This spacecraft is carrying a high resolution camera. I mean HIGH RESOLUTION, like 30cm/pixel. That is better that QuickBird images!

The image files are HUGE, for example full resolution image in JPG2000 format is about 1 GB. Picture size is 20 000 x 40 000 pixels.

As an alternative you can download image viewer and view them in lower resolution.

If you want to see these images go to the HIRISE site and have a look. Images are stunning and really interesting.

Don't expect to see any Martians there, they must be shy...

P.S. I'm sure Ray Bradbury is enjoying these beautiful images.

New Solid State Disk drive

Once upon a time I worked on a PC - mighty 386 CPU, 2 MB RAM and a floppy disk drive. Capacity of the floppy disks were whopping 1.2 MB. Common practice at that time was to create a RAM-DISK. In the RAM-DISK a part of main memory is allocated for a virtual floppy disk drive. Usually it was the same size as the physical floppy disk. Why? Floppy drives were sloooow and using RAM-DISK saved so much time during the day. Saving files on RAM-DISK was instantaneous. Of course, there was a downside. In the case of a power failure all data on RAM-DISK was gone. But in mosts cases I was willing to take that risk. At the end on the day just copy everything from RAM-DISK to actual floppy disk.

Now things are going back to that concept with a little twist. Today Pretec has announced a new SSD (Solid State Disk) with capacity of 64 GB (according to the TG Daily article. But on the Pretec's web site they have information for 16 GB SSD only.

I guess (hope) Pretec has not yet updated the web site with this new product. If they really do have a 64 GB SSD then I know what I want for Christmas this year!