Height Above Nearest Drainage goes mainstream in QGIS and ArcGIS

Between 2008 and 2016 Camilo Rennó and Antonio Nobre and co-authors published a series of papers describing "Height Above Nearest Drainage" (HAND), an approach for measuring the height above a mapped stream channel. Their approach basically involves identifying watersheds for each cell in a stream and relating that the to the upslope area through a join.  HAND is defined, more of less, as the difference between the elevation of a grid cell and the downslope elevation where it flows into the stream.

In 2011 I created an ArcGIS Toolbox called the "Riparian Topography Toolbox" and added a tool to calculate HAND. Height Above Nearest Drainage differs "Height Above River", which is calculated as the difference in elevation between a grid cell and a weighted average of grid cells within a stream. The two methods are conceptually similar although HAR uses multiple grid cells within a river.  Hence, it is more appropriate for deriving vertical height for LIDAR DEMs rather than typ

ical 10 or 30 meter DEMs.  HAND, on the other hand, is expected to be more computationally-efficient and more appropriate on coarser-scale DEMs.

As of ArcGIS version 10.7.1, I noticed that ArcGIS is now including a tool called "Flow Distance" in the Hydrology Toolset in ArcToolbox. This tool has an option for calculating distance in either horizontal or vertical distances. The vertical option appears to be equivalent to HAND so I decided to check it out.  I ran the "Flow Distance" tool on some of my study watersheds to see what the output looks like. The image on the left shows five sample watersheds with streams in blue, pour points as red dots, and watershed boundaries in black. The output from the Flow Distance tool is shown as a yellow to green to blue color ramp.

The image on the right shows the output from Flow Distance after thresholding into two classes: above and below 15 meters vertical height above stream channels. Purples are the areas below 15 meters and tan are the areas above 15 meters. You can see the some watersheds tend to have narrower riparian areas than others.

QGIS has an equivalent version of this tool.  It is called Terrain Channels and it also has a vertical component.

ArcGIS Flow Distance - https://desktop.arcgis.com/
en/arcmap/latest/tools/spatial-analyst-toolbox/flow-distance.html

QGIS Terrain channels - https://docs.qgis.org/2.18/ en/docs/user_manual/processing_algs /saga/terrain_analysis_channels.html

Rennó, C. D., Nobre, A. D., Cuartas, L. A., Soares, J. V., Hodnett, M. G., & Tomasella, J. (2008). HAND, a new terrain descriptor using SRTM-DEM: Mapping terra-firme rainforest environments in Amazonia. Remote Sensing of Environment, 112(9), 3469-3481.

Nobre, A. D., Cuartas, L. A., Hodnett, M., Rennó, C. D., Rodrigues, G., Silveira, A., & Saleska, S. (2011). Height Above the Nearest Drainage – a hydrologically relevant new terrain model. Journal of Hydrology, 404(1-2), 13-29.

Nobre, A. D., Cuartas, L. A., Momo, M. R., Severo, D. L., Pinheiro, A., & Nobre, C. A. (2016). HAND contour: a new proxy predictor of inundation extent. Hydrological Processes, 30(2), 320-333.

Dilts, T. E., Yang, J., & Weisberg, P. J. (2010) Mapping Riparian Vegetation with Lidar Data Predicting plant community distribution using height above river and flood height. ArcUser Magazine - https://www.esri.com/news/arcuser/0110/mapping-with-lidar.html

Finding the longest stream in each watershed using ArcGIS Network Analyst

Recently, I have found myself needing to identify the longest stream in a stream network for thousands of watersheds. Needless to say, doing this task manually is extremely time consuming.  Fortunately, there is a better way. Network Analyst in ArcGIS is a versatile tool that can be used to accomplish these tasks.  However, getting this workflow to work takes some finessing.  Here are the steps that I took.  Hopefully documenting these steps can help someone else.  All steps were completed in ArcGIS Desktop version 10.6.1.

Deriving the stream network:

In an arid environment, such as ours in Nevada USA, not all streams mapped by the USGS National Hydrography Network end up connecting to one another.  This can result in a disconnected stream network, which can make it really difficult to identify streams from headwaters down to pour points. I used the 1/3 arcsecond (10 meter) National Elevation Dataset Digital Elevation Model. I did the typical hydrological processing steps that included filling pits, calculating flow direction, and calculating flow accumulation.  I used a threshold of 40,000 cells to separate streams from non-streams.  The pour point

At this point I'll document the steps that I used to identify the longest stream in each watershed.

1.  Run the Feature Vertices to Points tool using the "BOTH" option.
2.  Run the Spatial Join tool specifying both the target and the join features as "vertices" from the previous step.
3.  This will create a new field called "Join_Count".  In the attribute table sort the Join_Count field, select Join_Count equal to 1 using a query similar to [Join_Count] = 1, and export the resulting selection to a new shapefile or geodatabase feature class. Name the new feature class "heads1" to represent the stream heads.
4.  Use the Snap tool in the Editing Toolbox. Snap the pour points to heads. I used a distance of 100 meters as my snapping distance.
5.  Use the Select by Location from the Selection menu at the top of the screen to identify "heads1" that intersect the snapped pour points.  Export the selection, name it "heads", and delete "heads1" from the previous step. This file represents stream heads but excludes pour points.


The image above shows the stream channels with each stream head mapped.

6. Network datasets exist within feature datasets within geodatabases. Create a new geodatabase, a new feature dataset within the geodatabase, and add heads, pour points, and streams to the feature dataset.
7. Create a new network dataset and add heads, pour points, and streams to the network dataset.  Keep the default settings, but don't allow turns or driving directions.

Geodatabase, feature dataset, network dataset with the channel heads, pour points, and streams participating in the network.


8.  Enable the Network Analyst extension (customize - extensions) and click on the Network Analyst Window from the Network Analyst Toolbar.
9.  Select "Create New OD Cost Matrix" from the drop down menu in the Network Analyst window.
10. In the Network Analyst menu right click on Origins and select Load.  Load the pour points.
11. In the Network Analyst menu right click on Destinations and select Load.  Load the heads points.
12. On the Network Analyst toolbar click the Solve button to identify all routes connecting pour points to channel heads.

The settings for the OD (origin-destination) matrix. The OD cost matrix using pour points as origins and channel heads as destinations.

13. In the Table of Contents right click on Lines in the OD Cost Matrix and export to a new feature class named "od_lines".

The origin-destination (OD) cost matrix depicts the distance from every pour point to every channel head as straight lines. However, the distances in the attribute table are in "stream network" distances and reflect network connectivity rather than Euclidean distance.

14. Run the Summary Statistics tool in the Analysis Toolset with OriginID as the case field and MAXIMUM of length as the summary field.

15. Join the summary statistics output table to the od_lines using the OriginID field.
16. Add a field called "Match" to the OD_lines shapefile and calculate it as [Total_Leng] = [MAX_Total_Leng]

17. Select records with Match = -1 and export as a new shapefile.

The image above shows the channel head that is furthest from the pour point of the watershed.

18. Use the Select by Location tool to identify channel heads that intersect the longest stream and export the feature class to the feature dataset that participates in the network dataset.

19. Right click on the network dataset and select Properties. Click Add and add the furthest_head feature class to the network dataset. Click OK.

20. Right click on the network dataset and click Build.

21. Add the updated network dataset into ArcMap.

22. In the Network Analyst window select "Closest Facility" from the drop down menu. Right click on Facilities and Load the pour points. Right click the Incidents and load the furthest_points.

23.  On the Network Analyst toolbar click Solve.

24. Right click onn Routes and click Data followed by Export Data to export the longest stream to its own feature class.

The image above shows the final longest stream from furthest channel head to pour point.

I'd be curious to hear if other people have more streamlined approaches to this.  I find Network Analyst to be quite efficient, but it involves a lot of steps.

Miscellaneous Hydrology Tools for ArcGIS - Find Longest Stream Path

Here is part four of the blog post on the Miscellaneous Hydrology Tools for ArcGIS. This post covers a tool called the Find Longest Stream Path tool. You can read the original blog post by clicking HERE or download the tool be clicking HERE. It is the fourth and final in this series about the Miscellaneous Hydrology Tools.

Surprisingly, there are no conventional tools in ArcMap that identify the longest stream in a watershed. However, many geomorphic metrics, such as relief ratio and watershed shape, are based on knowing what the longest stream is. In order rectify this situation I built a small model that calculates the longest stream. Unlike the other two tools in the Miscellaneous Hydrology Toolbox it does not provide an absolute answer.  For reasons unknown to me occasionally some watersheds get left out.  Nonetheless I feel that this tool is a useful and helpful addition that some people will enjoy having in their toolbox.

On the left is a sample of watersheds from central Nevada with the longest stream from each watershed shown in bold blue.  All streams are shown in light blue.  Watershed boundaries are in black. Pour points (those places where the stream exits the basin) are shown in green. Channel heads for the longest stream are shown in yellow.

There are four parameters for running this tool.  A input flow stream flow direction tool is required for understanding flow routing.  A standard flow direction raster can be clipped to the stream network to achieve this.  Input watershed polygons are required.  A temporary folder is required for storing outputs.  Finally, a dissolve field (ID for example) is required.  This should coincide with a field with the same name in the watersheds file.

On the right is a picture of the model.  Let me step you through how it works. For each stream flow direction cell the model calculates an upstream and downstream flow length. Using those flow lengths the model generates channel heads and pour points in each watershed. Using cost distance the model identifies the channel head with the greatest cost distance from the pour point. This becomes the end point. The combination of the end point and the pour point is used to create a least-cost path. Finally, grid cells are converted into flow lines.

Miscellaneous Hydrology Tools for ArcGIS - Trace Downstream Tool

Here is part three of the blog post on the Miscellaneous Hydrology Tools for ArcGIS. This post covers a tool called the Trace Downstream tool. You can read the original blog post by clicking HERE or download the tool be clicking HERE. Although there are tools in ArcGIS for tracing flows along vector networks there is no out-of-the-box tool for tracing downstream grid cells.  That is where this tool comes in.  Note that this tool uses a D8 flow algorithm. If there are diversions, like there might be in a vector-based hydrological network, then those won't show up.  The assumption is that water flows downhill and that it is controlled entirely by topography.

In the picture on the left we have the output from the Find Inflow Cells Tool described in my November 12th post HERE. You can use any point(s) that you are interested in tracing downstream flow from. This is just an example that fits into a recent workflow that I used and illustrates how the two tools might work in tandem. The small blue dots represent points that I want to trace downstream. These are inflow cells when this little part of the watershed was cut off from the remaining watershed (e.g. digitized line on the map).

The picture below shows the guts of the model as you would view it from ArcGIS Model Builder.  The Trace Flow Downstream is a fairly simple tool.  Using the flow direction from the previous tool and the points this simple tool takes a flow direction raster and reclassifies it into a backlink raster and then traces the path using the cost path tool. The reclassification is shown above with flow direction values in the left column and backlink values in the right column.

The picture on the right shows the results of the Trace Downstream tool.  Input points are the light blue points and red cells represent grid cells located downstream from those points. This tool could be used for a number of hydrological applications. For example, we may be interested in tracing a pollutant downstream. We may also be interested in identifying concentrated flow paths from overland flow. Finally we might want to use a tool such as this one to identify locations for placing erosion or pollutant control measures in order to maximize efficacy while controlling costs. For example, it might be most effective to place two to three measures in areas where flow is concentrated rather than dozens along the perimeter of the inflow area.

New tool - Miscellaneous Hydrology Tools for ArcGIS

I've got a new tool out.  It is a collection of miscellaneous hydrology tools called, appropriately enough, Miscellaneous Hydrology Tools for ArcGIS. You can download it HERE. This toolbox runs in ArcMap and performs three specialized functions that augment the standard hydrology toolset in Spatial Analyst. These tools include 1) Find Inflow Cells, 2) Trace Path, and 3) Find Longest Stream in a Watershed. In the next three posts I intend to detail how these tools work and how they can be applied to solve common problems.

Animation of the Mekong River - Tonle Sap flood pulse

Southeast Asia experiences dramatic swings in precipitation and river flows as a result of the Asian monsoon. Nowhere is this more dramatic than the Mekong River flowing through Cambodia and Vietnam. In Cambodia the mighty Mekong River winds its way across a vast flood plain. During the rainy season the flow of the Mekong increases so much that it reverses the flow of the Tonle Sap River increasing the size of the Tonle Sap Lake by 6 times or more. This animation shows the heart beat of the Mekong River, the annual flood pulse using data derived from Landsat satellites from 1990 to 2015. Some years are omitted due to cloud cover.

To view the animation on YouTube click HERE

Interesting finds in aerial photography

Every once in a while I encounter really interesting stuff when looking at digital elevation models and imagery in a GIS.  Today is one of those days. In the image below (40.8154 N 119.3856 W decimal degrees) the stream in the upper part of the image is flowing in two directions.  On the USGS topographic map it is named Rock Creek and flows to the south, but in the aerial photography it is quite evident that it also flows to the north at least some of the time.

In the image below I've zoomed in and see that a small jeep track may have caused an entire stream to shift from one basin into another (at least part of the time).

Interestingly, all of this is uphill and to the west of the cool and funky attraction known as Fly Geyser adjacent to the Black Rock Desert.

Nevada - last in the nation for LiDAR coverage

As Nevadans we're used to placing last or near last in lots of socioeconomic indicators. According to the USGS 3DEP program we may well be last in another category: LiDAR coverage.

LiDAR or laser altimetry is a valuable technology used for mapping all sorts of things ranging from building heights to flood zones to timber volumes.  I've used LiDAR a lot in my own work, and I believe that it is moving beyond a new cool tool to a critical piece of data infrastructure.  In fact, a lot of remote sensing products rely strongly on LiDAR.  As a state we need to be thinking about investing in LiDAR data if we want to solve the 21st century problems that all of the other states are solving.

Better flow accumulation with positive and negative weights

Most GIS users are probably familiar with calculating flow accumulation and many are probably also familiar with weighted flow accumulation. ArcMap, however, will not allow negative weights and this can be a severe limitation in some cases. Fortunately,  back in 2008 Bill Huber provided us with a simple yet elegant solution to this problem.  However, I've noticed that this problem tends to resurface from time to time on the forums. A while back I put together a geoprocessing model  to deal with this problem using Bill's solution which is available for download.

The map on the right illustrates how a tool, such as this one, might be used. I first performed the two operations that precede flow accumulation: filling sinks and calculating flow direction. In this case I used a 30 meter USGS NED DEM. Then using my new tool I used a weight raster that consisted of rain + snowmelt minus evapotranspiration. In areas with positive values water is being released into the environment. In contrast, negative values highlight areas where evaporation is  removing water from streams. In a typical flow accumulation operation flow accumulation only increases as one moves downstream. However, in arid environments we know that isn't true. Flows increase generally as one moves downstream due to multiple streams flowing into one another, but at some point flows begin to decrease because evaporation exceeds water inputs. The map above illustrates this in several places. Several streams flow out of the Ruby Mountains but many end when they flow into areas of high evaporation. Others flow off the map but  are diminished before they leave the map.

This example also demonstrates that streams could be treated as dynamic entities dependent upon both precipitation and temperature. The data for the map above was derived using the Climatic Water Deficit Toolbox for the month of June. Later in the summer months flows may be diminished and connectivity for fish species may be impacted. In contrast, during the wetter months streams may become more connected. Finally, I should note that for the one instance where I compared a result like this to an actual hydrograph it didn't stack up so well. The streams went dry mid-summer in the flow accumulation model yet actual hydrograph data showed greatly diminished but still existent flows due inputs from groundwater. Nonetheless, these sorts of dynamic flow accumulation models are likely to be better than standard static flow accumulation models.