Module 1- Applications in GIS- Crime Analysis

 A new term, a new course. We kicked off the Applications in GIS course learning about how GIS can be used in crime analysis. We made a choropleth map and a kernel density map depicting 2018 burglary rates in Washington DC. Then we created 3 hotspot maps (Grid Overlay, Kernel Density and Local Moran’s I) using 2017 homicide data for Chicago. We determined which analysis method was a better predictor of areas at higher risk of future homicide when compared to 2018 Chicago homicide data.

Crime density was highest within the Grid Overlay map, and lowest within the Local Moran’s I map. There was a correlation between crime density and total area of the of the hotspot, with Grid Overlay having the lowest total area since we set our parameters to consider only the top 20% of crime areas.

I did make an obvious mistake when I initially created my Grid Overlay map. I calculated the top 20% manually since there isn’t a tool for this task. There was no way to filter by attribute for this task either, but initially I used the select by attributes option and selected by objectID the last 62 features. I realized when I was completing my write up that even though I had sorted by homicide count in my attribute table, the object ID was a standard and would not have changed when I chose to sort by homicide count. There isn’t an option to select by the field number on the far left which would have been an accurate choice. I suppose I could have added a new field and numbered them that way but it was quicker to just manually select from 249 on towards the end after they were already sorted by homicide count. This was an obvious error, and I wasn’t sure how I overlooked it but fortunately I caught it because it would have greatly skewed my results. My initial (wrong) Grid Overlay had a significantly lower density than the other two analysis methods. One fact to consider when running these analyses is that the more manual tasks are involved in an analysis method, the more room there is for error.

Here are my 3 hotspot maps for the city of Chicago.

Module 6- Working With Geometries- GISProgramming

 This week we focused on working with geometries: reading geometries, working with multipart features and writing geometries. We got to work with the search cursor more but this time we worked with tuples which I found a bit more complicated when trying to print my script.

 

We worked with a rivers shapefile for this lab. Our first task was to create/open a new empty txt file that could be written in. The goal of this lab was to populate this text file with the name, x, y, coordinates, OID and to number the vertices for each row of the features in the rivers class.

We began with a search cursor. Initially when setting up my cursor I was not putting the search variables in the correct order. I put “NAME” first and when I tried to run my script I kept getting an integer error that cleared when I moved it to it’s proper order in the cursor.

 Once I got this sorted, I kept getting errors when trying to print the “NAME”. I tried using the getValue method that we used in the last lab to no avail. I kept getting an error that the “tuple” object has no attribute “getValue”. I also tried setting the NAME as a variable, like I did when I had issues in the last lab, but it wasn’t successful. I looked in the exercises we worked on before this assignment, but we never printed off any non-numerical variables, so I wasn’t sure how to approach this. I knew it was a tuple and I noticed as I reviewed last week’s script that last week’s search cursor was not so I tried variations on ways that can call a tuple value, but they didn’t work. Finally,  I referenced the documentation https://pro.arcgis.com/en/pro-app/latest/arcpy/data-access/searchcursor-class.htm and was able to print using the f’({})’ option they showed.

 

The goal was to populate a txt file, but to ensure I was printing the information properly without having to open the file each time I ran a matching print statement after my writing function. I focused on getting my print statement to work correctly while leaving the writing function commented out until I was ready to try to run it. When I finally got my print statement to print appropriately, I tried to move it to my txt file. I simply copied the entire parenthesis part of the statement and put it after my write to file function. However, I received an error that only one argument could be added. I was able to solve this by concatenating the individual statements as opposed to using a comma. After doing this I was able to run the script to add all the values that way, but my file was still blank when I checked to see if it filled in the correct data. I re-read the chapter on this and realized I left out my file “close” statement. I added the close statement, but I added it within the “for” loop instead of outside of it. This led to a value error that was resolved when I moved it outside of the loop. My txt file finally(!) populated but at this point it looked like it didn’t have the OID and Vertex numbers. I tried several options including str() before I realized it WAS giving me the first two numbers, but it was leaving out the space between them and the coordinates. I was easily able to resolve this by adding an extra space between each item I wanted in the write to file. I did not need this space for the print statement. It was very rewarding to see my populated text file.

 Here is a flowchart for my script along with an image of the resulting txt file.

                                    

Module 5- Exploring and Manipulating Data- GISProgramming

 This week we learned how to check for, describe and list data, create and populate a geodatabase with existing data in other folder, how to use a SQL statement within a python script using cursors, we reviewed lists and dictionaries and populated a dictionary from a selected feature class using a cursor.

Create and populate a new geodatabase

We began by creating a new geodatabase in our results folder and populating it with shapefile data provided in our data folder.  During this step I ran into some mistakes with my files early on. For both the creation of my geodatabase and the next step of populating my geodatabase I left out a “/” in my output. This led to me initially create a geodatabase named “resultsbnl26” in my module 5 folder as opposed to a geodatabase named bnl26 in my results folder. After I corrected this error, while populating my geodatabase I wound up creating a single shapefile named bnl26 to my results folder and nothing in my geodatabase folder. I realized quickly just how important that slash is.

Create a Search Cursor

We then were tasked to create a Search Cursor for the cities layer and to retrieve and print the name, population and feature of all cities that were listed as a “County Seat”. I created my cursor, ran my “for” loop and immediately had no issues printing my “name” statement but I ran into several snags trying to print the population. I kept getting an error that the program couldn’t concatenate an integer, so I tried converting it to a string, but I was not able to. I spent awhile trouble shooting and trying different methods of formatting my print statement but finally I had to set a separate variable to get the value of the population and then set that variable as a string instead of putting the getValue method within the print function. During this troubleshooting process I had a moment where I falsely thought I had figured out the population issue when, instead of an error, I got a syntax issue on the next line that wouldn’t allow my code to begin to run. I spent awhile wrongly troubleshooting that line (which was my feature print statement) until I finally removed it altogether to see if that fixed the issue. It simply moved the syntax error to the next line so at that point I knew that I still had an error in the previous line with my population print statement. This was a good reminder that sometimes the errors are not what or where we think they are.

During this section I also struggled to figure out how to use the newline character outside of a string since my previous experience with it had been inside a string and most of the options I was seeing online were within a string. I finally realized I needed to “+” it within the print statement rather than trying to tack it onto the end of my variable.

Create and populated a dictionary from feature class data

Our final task was to create and populate a dictionary that contained the names and population of each county seat. This task was my biggest struggle since we had only done the required tasks separately so far. I created my blank dictionary and I knew I needed a “for” loop and to somehow use the search cursor again. For some reason I tried to set my “for” loop as “for county in county_seats” (county_seats was my dictionary), and had a second “for” loop for the search cursor within the first for loop but realized that the first for loop was unnecessary. I knew how to update the dictionary but I kept accidentally populating my dictionary with [NAME : Pop_2000] (the titles of the rows) instead of getting the individual features names. At this point I realized I needed to add the getValue method to the line of code somehow, but I kept trying to add it within the “update” dictionary statement. I realized that I was making the same mistake I did earlier during the previous task and that I should set key and value variables that used the getValue method and then use those variables within the dictionary and it populated smoothly after that. I did run into a repeat of an issue I had earlier in the term where instead of printing off my final list it printed each iteration. I accidentally printed each iteration of my dictionary as it ran the “for” loop. This was easily corrected by adjusting the indent.

After I had completed all my tasks I went in and added the necessary print statements and retrieved messages for each task.

Here are the many screenshots of my final output as well as the flowchart I created based on my script.

 

Final thoughts

It was amazing to see the progress we have made in such a short time. I have really enjoyed learning to code, and I am disappointed that we are coming to the end of the term (only one week left). I am actively looking for ways to get more practice with these tools and I am hoping I can utilize them in my next few classes. One issue I am running into is that since it is all so new to me, I keep forgetting what options are even available to me. Another issue is discerning what variables to set for each task but this week I felt a lot more confident in my ability to “figure it out” when I hit a snag. I noticed that when I stepped away for a bit and came back to the task at hand fresh, I was able to solve my issues more quickly. This is a weakness of mine because I get very hyper focused on things I perceive as a puzzle and I don’t want to step away, but I am actively choosing to find a task I can do that forces me to take my mind off the “puzzle” of whatever error I am struggling with so that I can come back and see the code more clearly.

 

Module 4- GIS Programming- Geoprocessing

 This week we were tasked with using the coding skills we've gained over the past few lessons to run various geoprocessing tools in ArcGIS Pro. We used both the Model Builder feature and ran additional geoprocessing tools in Notebook. I love the functionality of the Model Builder and I could see how this would have been useful previously. I remember running a lot of individual tools during my Introduction to GIS final project and this would have more easily streamlined it (and possibly could have reduced some of unnecessary geoprocessing I did blundering around on my own for the first time. As someone who prefer visual context for tasks (and isn't GIS all about providing visuals?) this particular feature is very appealing and I look forward to using it more.

We built a simple model that clipped a provided soil shapefile into a specific basin area, then erased the areas that were classified as "Not prime farmland". This created the following output:

Our second task, in Notebook, was to add XY point data to a provided hospital shapefile, and then to run buffer analysis with the dissolve feature added. Although we worked through buffer analysis during the exercises, we had not added XY data to a shapefile before. We were told to reference the ArcGIS help pages for this task. I appreciated this because it gave me practice referencing online materials for solving new-to-me coding problems. I realize that after this class I will be constantly learning and will often need to reference internet resources in this manner. Even with the reference page I still find myself overanalyzing and second guessing (and therefore making mistakes) with my script. This seems to be a consistent theme for me in this class since I want to understand what each piece of the script is doing but with patience and practice I am understanding more and more. 

We are not supposed to share our actual codes on our blog for obvious reasons but here is a screenshot of my output in Notebook for this exercise:

And after my code was run it nicely populated directly onto my map as pictured: