Module 6- Working With Geometries- GISProgramming

 This week we focused on working with geometries: reading geometries, working with multipart features and writing geometries. We got to work with the search cursor more but this time we worked with tuples which I found a bit more complicated when trying to print my script.

 

We worked with a rivers shapefile for this lab. Our first task was to create/open a new empty txt file that could be written in. The goal of this lab was to populate this text file with the name, x, y, coordinates, OID and to number the vertices for each row of the features in the rivers class.

We began with a search cursor. Initially when setting up my cursor I was not putting the search variables in the correct order. I put “NAME” first and when I tried to run my script I kept getting an integer error that cleared when I moved it to it’s proper order in the cursor.

 Once I got this sorted, I kept getting errors when trying to print the “NAME”. I tried using the getValue method that we used in the last lab to no avail. I kept getting an error that the “tuple” object has no attribute “getValue”. I also tried setting the NAME as a variable, like I did when I had issues in the last lab, but it wasn’t successful. I looked in the exercises we worked on before this assignment, but we never printed off any non-numerical variables, so I wasn’t sure how to approach this. I knew it was a tuple and I noticed as I reviewed last week’s script that last week’s search cursor was not so I tried variations on ways that can call a tuple value, but they didn’t work. Finally,  I referenced the documentation https://pro.arcgis.com/en/pro-app/latest/arcpy/data-access/searchcursor-class.htm and was able to print using the f’({})’ option they showed.

 

The goal was to populate a txt file, but to ensure I was printing the information properly without having to open the file each time I ran a matching print statement after my writing function. I focused on getting my print statement to work correctly while leaving the writing function commented out until I was ready to try to run it. When I finally got my print statement to print appropriately, I tried to move it to my txt file. I simply copied the entire parenthesis part of the statement and put it after my write to file function. However, I received an error that only one argument could be added. I was able to solve this by concatenating the individual statements as opposed to using a comma. After doing this I was able to run the script to add all the values that way, but my file was still blank when I checked to see if it filled in the correct data. I re-read the chapter on this and realized I left out my file “close” statement. I added the close statement, but I added it within the “for” loop instead of outside of it. This led to a value error that was resolved when I moved it outside of the loop. My txt file finally(!) populated but at this point it looked like it didn’t have the OID and Vertex numbers. I tried several options including str() before I realized it WAS giving me the first two numbers, but it was leaving out the space between them and the coordinates. I was easily able to resolve this by adding an extra space between each item I wanted in the write to file. I did not need this space for the print statement. It was very rewarding to see my populated text file.

 Here is a flowchart for my script along with an image of the resulting txt file.

                                    

Module 5- Exploring and Manipulating Data- GISProgramming

 This week we learned how to check for, describe and list data, create and populate a geodatabase with existing data in other folder, how to use a SQL statement within a python script using cursors, we reviewed lists and dictionaries and populated a dictionary from a selected feature class using a cursor.

Create and populate a new geodatabase

We began by creating a new geodatabase in our results folder and populating it with shapefile data provided in our data folder.  During this step I ran into some mistakes with my files early on. For both the creation of my geodatabase and the next step of populating my geodatabase I left out a “/” in my output. This led to me initially create a geodatabase named “resultsbnl26” in my module 5 folder as opposed to a geodatabase named bnl26 in my results folder. After I corrected this error, while populating my geodatabase I wound up creating a single shapefile named bnl26 to my results folder and nothing in my geodatabase folder. I realized quickly just how important that slash is.

Create a Search Cursor

We then were tasked to create a Search Cursor for the cities layer and to retrieve and print the name, population and feature of all cities that were listed as a “County Seat”. I created my cursor, ran my “for” loop and immediately had no issues printing my “name” statement but I ran into several snags trying to print the population. I kept getting an error that the program couldn’t concatenate an integer, so I tried converting it to a string, but I was not able to. I spent awhile trouble shooting and trying different methods of formatting my print statement but finally I had to set a separate variable to get the value of the population and then set that variable as a string instead of putting the getValue method within the print function. During this troubleshooting process I had a moment where I falsely thought I had figured out the population issue when, instead of an error, I got a syntax issue on the next line that wouldn’t allow my code to begin to run. I spent awhile wrongly troubleshooting that line (which was my feature print statement) until I finally removed it altogether to see if that fixed the issue. It simply moved the syntax error to the next line so at that point I knew that I still had an error in the previous line with my population print statement. This was a good reminder that sometimes the errors are not what or where we think they are.

During this section I also struggled to figure out how to use the newline character outside of a string since my previous experience with it had been inside a string and most of the options I was seeing online were within a string. I finally realized I needed to “+” it within the print statement rather than trying to tack it onto the end of my variable.

Create and populated a dictionary from feature class data

Our final task was to create and populate a dictionary that contained the names and population of each county seat. This task was my biggest struggle since we had only done the required tasks separately so far. I created my blank dictionary and I knew I needed a “for” loop and to somehow use the search cursor again. For some reason I tried to set my “for” loop as “for county in county_seats” (county_seats was my dictionary), and had a second “for” loop for the search cursor within the first for loop but realized that the first for loop was unnecessary. I knew how to update the dictionary but I kept accidentally populating my dictionary with [NAME : Pop_2000] (the titles of the rows) instead of getting the individual features names. At this point I realized I needed to add the getValue method to the line of code somehow, but I kept trying to add it within the “update” dictionary statement. I realized that I was making the same mistake I did earlier during the previous task and that I should set key and value variables that used the getValue method and then use those variables within the dictionary and it populated smoothly after that. I did run into a repeat of an issue I had earlier in the term where instead of printing off my final list it printed each iteration. I accidentally printed each iteration of my dictionary as it ran the “for” loop. This was easily corrected by adjusting the indent.

After I had completed all my tasks I went in and added the necessary print statements and retrieved messages for each task.

Here are the many screenshots of my final output as well as the flowchart I created based on my script.

 

Final thoughts

It was amazing to see the progress we have made in such a short time. I have really enjoyed learning to code, and I am disappointed that we are coming to the end of the term (only one week left). I am actively looking for ways to get more practice with these tools and I am hoping I can utilize them in my next few classes. One issue I am running into is that since it is all so new to me, I keep forgetting what options are even available to me. Another issue is discerning what variables to set for each task but this week I felt a lot more confident in my ability to “figure it out” when I hit a snag. I noticed that when I stepped away for a bit and came back to the task at hand fresh, I was able to solve my issues more quickly. This is a weakness of mine because I get very hyper focused on things I perceive as a puzzle and I don’t want to step away, but I am actively choosing to find a task I can do that forces me to take my mind off the “puzzle” of whatever error I am struggling with so that I can come back and see the code more clearly.

 

Module 4- GIS Programming- Geoprocessing

 This week we were tasked with using the coding skills we've gained over the past few lessons to run various geoprocessing tools in ArcGIS Pro. We used both the Model Builder feature and ran additional geoprocessing tools in Notebook. I love the functionality of the Model Builder and I could see how this would have been useful previously. I remember running a lot of individual tools during my Introduction to GIS final project and this would have more easily streamlined it (and possibly could have reduced some of unnecessary geoprocessing I did blundering around on my own for the first time. As someone who prefer visual context for tasks (and isn't GIS all about providing visuals?) this particular feature is very appealing and I look forward to using it more.

We built a simple model that clipped a provided soil shapefile into a specific basin area, then erased the areas that were classified as "Not prime farmland". This created the following output:

Our second task, in Notebook, was to add XY point data to a provided hospital shapefile, and then to run buffer analysis with the dissolve feature added. Although we worked through buffer analysis during the exercises, we had not added XY data to a shapefile before. We were told to reference the ArcGIS help pages for this task. I appreciated this because it gave me practice referencing online materials for solving new-to-me coding problems. I realize that after this class I will be constantly learning and will often need to reference internet resources in this manner. Even with the reference page I still find myself overanalyzing and second guessing (and therefore making mistakes) with my script. This seems to be a consistent theme for me in this class since I want to understand what each piece of the script is doing but with patience and practice I am understanding more and more. 

We are not supposed to share our actual codes on our blog for obvious reasons but here is a screenshot of my output in Notebook for this exercise:

And after my code was run it nicely populated directly onto my map as pictured:

Module 3- GIS Programming- Debugging and Error Handling

 This week’s assignment focused on diagnosing errors and debugging scripts. We learned several different types of errors and techniques for identifying errors and debugging, notably using try-except statements.

We were given 3 error-filled scripts to fix in IDLE. For the first script we had to identify and correct syntax errors. I fixed the first one easily but got stuck on one for quite a bit.  Without giving away the answer, I was stuck trying to figure out what was wrong with the variable for quite awhile before realizing I was misunderstanding how the words in a loop worked.

I was able to work through most of the errors in the second script methodically but ultimately, I ended up with the script running and giving me the appropriate output, but still giving an error. This took a while for me to resolve and after trying several things that only made the script not run, I realized that it was because I still had the script open in ArcGIS Pro. Once I closed the program the script was able to run and give me the requested output.

The third script required us not to directly fix the error but to add a try-except statement that allowed the script to run but to print the error instead of terminating the script. I was able to quickly identify where the issue in the script was and where to put the “try” statement but had a more difficult time figuring out where to put the “except” statement. I ended up adjusting my statement multiple times before I was able to get the desired result. The following flowchart shows the steps I took to debug this particular code.

Here are copies of the output of my debugged scripts:

Module 2- GIS Programming- Python Fundamentals

 This week we read about python fundamentals including: if, elif and else; lists, dictionaries and tuples; commonly used methods and function; loops and Booleans. We then practiced implementing these fundamentals both in the practice exercises and module 2 lab.

We were assigned 4 tasks: to print our last name from a created list, to fix errors within a prewritten code set, to populate and empty list with 20 random integers using a while loop with a counter variable. Then to use an “if” statement and a while loop to remove an “unlucky” number of our choice from the previously created list and then reprint the list with this integer removed.

 The first two tasks went smoothly. I created a string of my name and used the split function to create a list of individual names. Then I was able to use the index function to print only my last name. The second task was to correct errors in a provided “for” loop.

The third task was more complicated, but I feel like I learned a lot through this process. I began by creating my empty list and then made sure to import the random module, and began to put together my "while" loop. For my initial code I utilized the “random” function too soon in the script. This resulted in my output simply making a list of 20 duplicate integers. The second script I ran worked perfectly and gave the desired output, but it did not contain a counter variable which was a requirement of the assignment. I wasn’t quite grasping the concept of a counter variable and breaking a loop, so my next script ended up being an infinite loop and crashed the program. Finally, after a bit of re-reading I created a working script that contained all the desired components, however, my output was printing each addition to the list individually as a separate list, so I ended up with a giant triangle of 20 lists each with a single additional number. This was because I accidentally put the “print” function indented under the while loop. By moving it to the left where it belonged and not including it in the loop, I was finally able to print a single list of 20 randomly generated integers.

The final task built upon task 3. We were to remove an “unlucky” number (integer of our choice) from the list generated in the last task. We were to include an “if” statement to print one of two statements. If the integer was absent, we were to print that it was not in the list. If it was present we were to print a statement saying that the integer would be removed from the list for the total count of times it was included in the list. We were then to use a “while” loop to remove the integer from the list for the “count” of times it was present. I initially typed my print statements in my “if/else” statement with the integer itself in the string but I quickly realized I should reference the variable (unlucky is what I designated it) as a string within the code in order to create a code that was more efficient and could be changed by simply changing the variable if I desired, so I made sure that instead of typing “9 is not in the list” I would make sure the “9” was actually str(unlucky). I tested this by changing the variable to another integer and the code ran smoothly and changed easily along with the newly generated list and newly chosen integer. I finally was able to print the list with the number removed and completed the assignment, here is an image depicting my final output followed by a flowchart showing the steps I took to complete the task.

Module 1- GIS Programming- Python Environments and Flowcharts

 The first week of GIS Programming has commenced. I was nervous going into this class as I have no coding experience, but I am excited to learn something new. This week we learned the very basics of python and practiced making flowcharts. Our first task was to run a pre-scripted code that created all the folders we would need for the course. As I looked at the code, I noticed how easy it would be to slightly alter this code to use for other courses, assuming one was following a consistent folder pattern for each module. I find the functional aspect of coding interesting because I haven’t understood or considered all the different tasks it can be used to streamline before now.

One aspect of coding I think is less intuitive to me at this point is knowing what values to define. I could easily see myself forgetting to define a value when setting up my codes, especially as the codes get progressively more complex. I presume that it will become more instinctive with time/practice. In the case of this week’s assignment, which was to create a flowchart for a simple math calculation, it was such a different way of looking at a simple math problem- having to input each piece separately before putting it all together. When I began putting my flowchart together, I was unsure if “pi” would be a known or unknown variable in python so decided to test it by running the code to determine degrees based on my flowchart. It told me that there was an “error” with variable the variable “pi”. I tried again but added that pi=3.14159 and it gave me to output so I made sure to define that variable to my flowchart. I had run a google search before “testing” my equation and found that there is some way to use a math module or library that would allow me to simply use “pi” without declaring it but for the purposes of this lab didn’t investigate it further since we weren’t tasked with running the code at this time. However, I do want to follow up and figure out how that works at a later point if it is not covered in future labs.

A flowchart depicting the calculation of degrees when radians=3

Our last task was to access and read “The Zen of Python” and share what we think it means. I read “The Zen of Python” as a set of guidelines to consider when developing scripts using this program. It makes clear that the developers of python value clarity (with statements such as “explicit is better than implicit”), and simplicity (by defining a simple>complex>complicated hierarchy, and a preference for sparse over dense and flat over nested). This work makes clear that the purpose of python is not flashy/overcomplicated scripts but rather the beauty of python lies in ease of use. Fortunately for me, this challenges my previously contrived ideas of what coding would look like as I imagined it to be a very difficult, complicated process.

Module 7- Cartography- Google Earth

 This week focused on converting map layers to KML files and uploading to Google Earth as well as creating a Google Earth Tour. The tour was a neat feature I'd never experimented with. For our map and tour we focused on south Florida.  The map created was a dot density map with surface water layers added. Before we uploaded our layers to Google Earth we practiced using the Layer to KML tool to convert the layer into a format that could be used in Google Earth. We also learned how to organize layers and add images in Google Earth. 

The tour was an overview of major Florida metropolitan areas. After creating placemarks for each area I wanted to feature, I gave an view of the entirety of south Florida and the zoomed in and out on the placemarks then panned and zoomed in, out and around the downtown areas. 

Here is a copy of my Google Earth map.