Module 11 - Sharing Tools

This was our last assignment of the course, and in this one we work with Python tools again. Last week we learned to create tools, and this week we learned to share them. This is somewhat of a continuation from last week. In this lab we modified script so that the parameters would use file paths set in the tool itself instead of from the set file paths in the script. We did this by using the sys.argv[] code. The positions numbers for this start at "1", as opposed to the arcpy.GetParameter() code, which starts at "0".

We learned to edit the item description of the script tool in ArcCatalog. We learned to embed the script into the tool by right-clicking the tool in ArcCatalog and selected "Import Script." One of the more interesting features we can do now that we've imported the script is we can now set a password. The password still allows the tool to run, but the password needs to be input to view or edit the script. The tool in the assignment creates a number of random points within a feature, and then creates a buffer around those points. Below are the results of both my tool dialog and the results in ArcMap.

You can see the feature in purple with the random points (in this case, 50 of them) in black, with the light blue buffers around them.

I really enjoyed this course, and I learned a lot from it. I would have liked to have done a little more of script writing, but this course is designed to use Python with ArcGIS, and I think it teaches that very well. I learned how to use Python to perform geoprocessing tasks, and about toolboxes and tools within Python, working with rasters and different geometries, etc. I think this course is a great start as one moves forward to work more with GIS and Python programming.

Participation Assignment #2

The article that I found is an interesting application to GIS. This describes how, in 1996, TransCanada used GIS to select the best route for an expansion pipeline. I found it interesting because this seems to be one of the first times this type of analysis took place using GIS. It's a route selection analysis very similar to what we did early in the semester in the other course with the corridor analysis. They set the project goals, which were threefold. First, minimize risk to public safety. Second, minimize environmental impact. Third, minimize construction and operation costs.

This particular project used information from already existing digital and map sources and interpreted imagery. They used GIS techniques to include “exclusions, constraints, and opportunities.” When minimizing environmental impacts, they weighted environmentally sensitive areas with heavier constraints. Collocations with already existing rights-of-way were weighted with lower constraints, so the GIS could label them as opportunities. They also weighted river and road crossings heavier, as that would increase construction costs. They also created buffers around urban areas to minimize the impact on the population.

Using these criteria, TransCanada developed several route alternatives. The GIS was able to provide the lengths of the routes and detailed crossings information. They also analyzed the impacts of the different criteria on the various alternative routes. Before selecting a preferred route, more data were gathered based on aerial imagery and ground-truthing. This method became a go to method for other entities for route selection, and it is very similar to scenarios we have seen in the other course.

GIS-based process helps TransCanada select best route for expansion line

Module 10 - Creating Custom Tools

In this assignment, we learned to create custom tools and toolboxes. First, we were create a script tool from a provided script, which was to run in ArcMap. I opened the script and modified the script name, last modified date, and modified by comments, and saved it under my name. In ArcMap (ArcToolbox) I created a new toolbox, and within that toolbox, a new script. Creating a new toolbox was rather straightforward, was just right-click -> New Toolbox; very similar to making a new folder. Next we were to set parameters. Creating the parameters is pretty simple tool, as in Properties there is a tool dialog box that makes it easy to not only create parameters, but to modify their settings (data type, multivalue property, etc.). Below is a screenshot of my script tool window.

Now that the parameters are set in the script tool, they needed to be set in the script. I changed the original four variables in the script, replacing them with the arcpy.GetParameter() command. I had a little bit of an issue here, as I was using the arcpy.GetParameterAsText() command initially. This didn’t work, as this only reads parameters as text, which caused errors. As the outputFolder can’t be concatenated with strings, I used the str() function on the outputFolder variable, and ran the tool. Next we were to replace the print statements with arcpy.AddMessage() statements. Initially, I did this and no matter what I did, I could not get the code to work when running it. The problem turned out not to be in the code. What was going on is that I was trying to run the script in PythonWin as a stand-alone script. The arcpy.AddMessage() statements are, I think, designed to be run as a script tool. When I opened ArcMap and ran the tool again, it worked fine. Below is a screenshot of my results window.

To share the toolbox, tool, and script, I selected both the toolbox and the script and sent them to a compressed (zipped) folder.

I feel I learned a lot in this lab, and I think I’ll get a lot of use out of using toolboxes and tools in the future. I think one of the most valuable components of this lab is the idea of sharing the tools and toolboxes. Once I figured out the couple of issues I had and why what I was doing was causing problems, the assignment seemed pretty straightforward. Creating toolboxes is rather user-friendly and should be picked up quickly by most people familiar with Windows. The same can be said for setting the parameters in the dialog box, although it’s important to get the settings such as “Multivalue” correct, or the tool won’t run properly; it is easy to go back and check those, though. Once I understood what the arcpy.GetParameter() and arcpy.AddMessage() functions are doing, I realize they are quite useful and I’ll try to find ways to use them in the future. I also like the idea of sharing toolboxes and scripts, so others can use or modify them.

Module 9 - Working with Rasters

This assignment taught us to work with rasters using Python. We worked with 2 rasters, elevation and landcover, to create a Python script is a single raster image with the following criteria: 1) Forest landcover classification; 2) slope between 5 and 20 degrees; and 3) aspect between 150 and 270 degrees.

It is becoming almost habit to start off by importing the necessary modules needed for the script and to set the overwrite output to true. For this assignment, I attempted to create the output in a .tif format, but I got several errors, so I went with the optional part and created a file geodatabase. For the rest of the script, the module exercise was extremely helpful and relevant to the assignment. I created a conditional statement so the script would only run if the Spatial Analyst extension was available. Further down toward the end of the script, there is an else statement printing a message if it is not available. Within the if statement is where the bulk of the script is located, as I only want it to run if the spatial analyst module is available. I used the remap value and reclassify tools to reclassify the landcover raster so that only the forested areas are given a value of "1". I assigned the elevation raster to a variable and used the slope and aspect functions to create temporary rasters of those types. I was impressed with how user friendly the map algebra commands are when used in Python, so performing the slope and aspect calculations weren't an issue. I created temporary rasters based on the 3 criteria listed above and using "less than" and greater than" statements in the script. I combined the 5 rasters (four slope and aspect rasters and the reclassified landcover raster), and saved it to make it a permanent raster. Below is a screenshot of the raster, with the "0" (reddish-brown in color) signifying areas of the landcover raster not classified as forest and areas labeled as "1" (the green areas) showing areas that are forest land cover.

This assignment was far less stressful for me than last week's assignment. I feel I have a greater understanding of this concept, and the exercise was extremely helpful, so if I had a syntax error I could look at the exercise (and the textbook) to determine if I was missing a parentheses or whatever the error was. The only issue I had dealt with the file geodatabase I created. Even setting the overwrite output to true does not allow you to overwrite the geodatabase, which I suppose makes sense. The geodatabase has files within it, and you usually can't overwrite a non-empty folder. I was unable to find any workaround, so after running the program once, you still have to go into the Results folder and delete the geodatabase. I didn't do this as it just occurred to me while writing this, but I wonder if it's possible to have the code delete the files within the geodatabase at the end of the script, and then have Python overwrite the empty geodatabase. I liked working with rasters in this assignment and I was impressed that Python almost seems to be made to work with some of the map algebra operators. I like that a relatively short and simple code can do this much geoprocessing. On another note, the ELearning site appears to be down, so I chose to email the instructor the relevant deliverables.

Module 8 - Working with Geometries

This week’s lab dealt with working with geometries in Python, and it was a challenging one for me, partially because working with arrays and nested loops are not my strong suit and partly due to a pretty intense work schedule this week. In this lab, we were working with the “rivers.shp” file, using loops to iterate through the cursor, and we wrote the data to a .txt file.

After importing the modules and setting the workspace, I created a search cursor for the shapefile I was working with. This called the OID field, the SHAPE geometry object, and the NAME field. I created a .txt file using the write mode, following examples in the text and in our exercise. To write the data to a text file, I needed to create a couple of “for” loops. The first iterates through each row in the shapefile’s attribute table. I created a variable called vertexid within the loop set to 0, so that it resets for each new feature. Within that loop, I created a second loop using the .getPart() method, which iterates through each point in the array of the row. The .getPart() method allows me to access the points in each array. If I did not use this method, another “for” loop would be necessary. I added “1” to the vertexid variable in this loop to keep track of the number of points in each feature.

Here’s where it got tricky for me. I needed to use the write() method to write to the .txt file. I wanted to add the Feature/row ID, the vertex ID, the X and Y coordinates, and the name of the river feature, using “\n” to ensure each vertex is on its own line. I ran into a wall here with syntax errors and exceptions at first. I was pretty sure the loops were correct (at least mostly) because I immediately was able to print correct values for the X and Y coordinates. I ran into a syntax error which gave no input into where or what the error was, so I started deleting code (remembering later I could have commented it out) and attempted to run the program. The issue turned out to be a print statement I had created underneath the write() statement, which wasn’t necessary, so I just left it out. The other error I had was not using arrays properly. I had the output in the .txt file correct, but the name and feature were printing “NAME” and “FEATURE”. Once I changed this to row[2] in the write() statement, the program wrote to the .txt file correctly. The output is shown below.  The first column shows the Feature/row ID, the second column shows the vertex ID for that feature (notice how it resets for each feature), the 3^rd and 4^th columns are the X and Y coordinates of the vertex, and the 5^th and 6^th columns are the feature name and type.

This was a really good lab, as I learned about working with geometries and got more experience working with arrays. Both the instructor and the other students on the discussion boards were a big help with steering me in the right direction with my code. I wish I had had more time to really work with it, as opposed to an hour or two at a time, but even so, I learned a lot and I think it’s less confusing and less intimidating to work with than with FORTRAN or even IDL code.

Module 7 - Explore/Manipulate Spatial Data

In this assignment, we learned to explore and manipulate spatial data in Python. I found this to be a great introduction to manipulating spatial data and I learned how to use some functions that I had no experience with. The objective was to create a script that created a new geodatabase, copied data from the data folder into the new geodatabase, and populated a dictionary with the names and population of county seats in New Mexico.

First I imported the modules and set the environments so that the output could be overwritten. Then I created a new geodatabase and set my current workspace. I needed to create a variable returning a list of all feature classes in the workspace folder. I used a “for” loop to copy feature from the workspace to the new geodatabase. Here, I made a simple mistake that took a little bit to find. I inadvertently set the workspace and the new geodatabase to the same location, so I had to correct that. The new files use the basename property from the arcpy.Describe() function. I then created a search cursor for the cities layer. It took a while to get the syntax correct, and I tried some different options with delimiting the FEATURE field. This part of the lab was tricky with getting the single and double quotation marks correct. Then we started working with dictionaries. I created an empty dictionary, which I was to then populate. This step took me the longest, and the issue was mainly just figuring out the proper syntax to do what I wanted to do. I wanted the county seats and their populations, but I had to select the county seats from the cities field, which added some complexity to the script. I added print messages to each step and made sure to comment the script.

Looking back over the exercise for this assignment was a big help, as sometimes I couldn’t remember the syntax exactly or which command to use in my script. We were provided a template, which was helpful for the most part. It was helpful to see in which order we were to perform a step, but sometimes when the code for that step was only one line I wondered if I was missing something. I also have a comma in my dictionary instead of a colon separating the fields in the output; I’m not sure what’s causing that, but the code does work without error. Below is a screenshot of the output (no actual code).

Module 6 - Geoprocessing with Python

In this lab, we were to write a script that performs some geoprocessing functions. For me, being able to use the Search tool to look up the proper syntax for a geoprocessing function was very useful. In this script, I imported the relevant modules and set the workspace. This script first adds XY coordinates to the "hospitals" shapefile using the AddXY tool. It then uses the Buffer tool to create a 1000 meter buffer around the hospital features. After the buffer is created, it uses the Dissolve tool to dissolve the hospital buffers into a single feature. After each geoprocessing function was completed, I used the GetMessages() function to print out the messages from that tool. After that I went back through and commented the script to show which geoprocessing tool was being run. I also went back to the top of the script and used comments to add my name, date, the script's name, and a description of what the script does. Below is a screenshot of the results from the interactive window (created from the GetMessages() function).

This lab, while relatively short, was a good instruction of how to use Python to perform various geoprocessing tools, and learning that the Desktop Help and the Search button in ArcMap will show us the syntax is very helpful, and will be useful in future labs.

Module 5 - Geoprocessing in ArcGIS

This lab was about geoprocessing in ArcGIS. In it, we learned to create a toolbox, and how to create tools using ModelBuilder and scripts tools. We also learned to export scripts from ModelBuilder. We also learned how to update a script derived from a model so that it will work on its own.

This module was really interesting to me because I have wanted more practice with ModelBuilder since we were introduced to it last semester. I created a new toolbox, which was a pretty straightforward process in ArcCatalog. Inside the toolbox , I created a new model, the end product of which would contain the soils within a basin not classified as “not prime farmland.” The model uses the clip, select, and erase tools to accomplish this.

I set all the input and output variables as model parameters. This allows us to change the input features, output name, or filepath if we choose. One thing to note, which I feel is pretty easy to forget, is that all the input features need to be added to the current document for the model and script tools to work properly. We exported the model as a Python script. This is really useful as now a lot of the scripting is done for us. But there was still some to do to allow the script to work as a stand-alone script. To do this, all we really needed to change in this case was to explicitly define the full file path of the layers we were working with. It’s also important to allow Arc to overwrite the output, or there will be an error and the script will not run. Using this process, I created a shapefile showing the soils not classified as “not prime farmland.” A screenshot of the shapefile is below.

Next we learned how to share a toolbox. I selected both the toolbox and the scripts folder and sent them to a new zipped folder.

This lab, while not very long, was very informative and full of useful material. I really liked how easy it was to export the model to a script, and to create the model itself. I also really liked how complete the script was; it even had some comments to describe what the script was doing. I learned quite a bit in this lab and hope to work with some of this in the future.

Lab 4 - Debugging and Error Handling

This lab was about learning to debug and handle errors in Python code. We also got some experience with the debugger in PythonWin. I think it’s really helpful to learn some of this before writing code too much more complicated. In this lab, we had three scripts with which we had to handle errors in different ways.

Part I was basically finding and correcting errors, so that the code would print out the field names of the “parks” shapefile. In this case, we knew we only had 2 errors, which of course when we write a code we won’t know. Below is a screenshot of the output of the first corrected script.

The second script had a total of eight errors, most of which were pretty easy for me to spot. This script was to print the layers active in a data frame. This had several errors, some of which were in commands we have not used yet, but using a search engine to look up the command in question (mainly to see what the syntax normally looked like) was very helpful. Below is a screenshot of my output.

The third part was trickier. We were to bypass the error using a try-except statement, where Part A returns an error message, but Part B runs correctly. This was mainly trial and error for me as far as where to place the try-except statements. I imagine I will get faster at it with more practice. Below is a screenshot of my output.

I feel having some prior experience with debugging helped a lot. The new concepts to me were really learning how to use the debugger tool for Python and using the try-except statements. I feel this was a good lab to help us get comfortable with the types of errors we will see in future labs.

Participation Assignment #1 -

When looking through the UWF library, I found an article relating the use of GIS to establish relationships between precipitation and terrain data. The article was written by Hong Haoyuan from the Jiangxi Meteorological Observatory in China. The objective of the study I chose is to use GIS and a regression equation to provide better support for flood disaster prevention and support.

The authors obtained DEM, slope, slope aspect, and water system data. Most of the data came from the Jiangxi Meteorological Bureau; precipitation data came from observational data from several meteorological stations from the days before, during, and after the rainstorm event.

The authors used a multiple linear regression model based on the least square method. They determined that the greatest impacts on the water system were due to rainfall, slope, altitude, and slope aspect (the direction the slope faces). Using the analysis tools of ArcGIS, they found the distribution characteristics of the criteria, and found that the factors most related to flood disasters in this region are the slope and slope aspect of the terrain. The province being discussed in the study has flat plains in the center and more mountainous regions in both the western and eastern sections. This region contains Poyang Lake, the largest freshwater lake in China, and the precipitation amount showed a close correlation with proximity to the lake.

Also using the tools in ArcGIS, the authors mapped the cumulative rainfall distribution of the event, with a spatial resolution of 30 m x 30 m. Based on the spatial distribution of precipitation with the maximum rainfall near the areas of steeper slopes and near the large lake, they determine that the orographic lifting process was an important contributor to the rainstorm event. Orographic lifting is air forced upwards by terrain, causing it to cool and become saturated and produce rain. In addition to larger precipitation near the large lake, there were also precipitation maxima just off some of the steeper slopes, supporting the authors idea that orographic lifting was a major contributor for this rainstorm.

The authors used observational data from weather monitoring stations to determine which factors were the most important to the spatial distribution of rainfall and established a regression model to examine this. Using ArcGIS software, they analyzed the spatial distribution of rainfall using those criteria. Using their model, the modeled rainfall and actual rainfall differed by only 1.358%, so GIS showed that using their criteria produced an accurate representation of rainfall distribution in this case.

I chose this article because this is an interesting blend of GIS and meteorological concepts. I found it really interesting to see some of the meteorological concepts being mapped out using some of the GIS analysis techniques that we have recently learned. I feel that GIS isn’t used enough in meteorology, and I love finding interesting articles that blend the two together.

http://search.proquest.com.ezproxy.lib.uwf.edu/environmentalscience/docview/1553423204/F6880D76EFCE4AA3PQ/17?accountid=14787

Module 3: Python Fundamentals Part II

This week we learned some more about the fundamentals of Python. We learned how to import modules into a script, find and correct errors in the code, create loops and conditional statements, add comments, and how to iterate variables within loops.

For this assignment, we were provided a partially written code that we needed to complete. The objective was to create a small dice rolling game based on a players' name length, then create a list containing 20 numbers. We needed to create a conditional statement with a nested while loop that removed all instances of a specified number from that list.

First, we added a line importing the random module. Then we were directed to a couple of lines of code with errors in them that we needed to find and correct. Then the fun part of writing new code began.  The next part of code needed to create a loop that generated 20 random integers between 0 and 10 to a list. I started with a counter and an empty list, and used a while loop to accomplish this. The next block of code took some more thought. I wanted to remove a specific number anywhere it turned up in the list. I assigned a new variable and assigned the integer I wanted to remove to that variable. Using a conditional statement and the .count ( ) method, I determined if that specific number was contained in that list. If it did, I used a while loop to find how many times and removed that specific number from the list. Throughout the writing of the program, we were encouraged to use comments, as they not only tell who wrote the script and when, but they also can help describe in plain English what a line or block of code is supposed to accomplish. The screenshot below shows the output of my program. It shows each player's dice roll and whether they won or lost, the 20 integer long list, how many times my specific number was removed, and the list after that integer has been removed.

I really enjoyed working on this lab. It was good to start writing some simple scripts while practicing importing module, using conditional statements and loops, and using comments within the script. I think I may have had an easier time than some due to previous programming experience, but this was an excellent exercise and I still learned a lot about Python specifically. I'm still used to code not being happy with me when I leave an open-ended conditional statement (one without endif), but I am enjoying learning Python and I like it much better than what I have worked with before. I enjoyed working on this script and I look forward to writing more.

Module 2 - Python Fundamentals Part I

In this module, we're still getting our feet wet with Python programming. We learned about string variables, methods and functions used to manipulate strings and lists, and how to perform basic math functions with Python.

After working through the weekly exercise which got us familiar with strings, lists, functions, and methods in Python, we began the assignment, which was just to write our first Python code. In this lab, we create a string of our full name, and then split that string into a list (First, Middle, Last). We used indexing to print our last name from the list. Then we had to assign a couple of variables. The first variable uses the len function to find the length of our last name, and the second variable multiplies the length variable by 3. When run successfully, the code should display our last name and the value that is three times the length of our last name.

As you can see, my last name contains 8 letters, and 8 *3 = 24, so the code executed correctly. I enjoyed working with the basics of Python and am interested in learning more. As someone familiar with FORTRAN, I can already say I am enjoying Python a lot more.

Module 1 - Introduction to Python

This was our first week in GIS Programming, and the first assignment is mainly intended to get us used to the software and editors that we will be using to create Python scripts. Python is a programming language created in 1991 by Guido van Rossum and, as of version 10.0, is the primary scripting language used in ArcGIS. We were introduced to Python 2.7, including the PythonWIN editor and IDLE. We were also introduced to pseudocode, which is language similar to English that allows us to represent a program concisely.

Our main assignment this week was to run a script using PythonWin. The script creates folders for all 12 modules in our GIS Programming directory, as well as 3 folders with each module folder.  This was a rather simple task. I had an error the first time I attempted it as I had already created a folder for module 1, but removing that folder and rerunning the script easily fixed that error. Below is the screenshot of the folders that the script created. You can see all 12 module folders and on the right, the 3 folders within Module 1.