When Web Services Go Bad

By , January 2, 2012

A few years ago I published a number of articles around using the National Weather Service NDFD web service to get current weather conditions and forecasts.  Unfortunately a few months ago quite a few folks started mentioning the example applications had stopped working.  I finally got time to look at the problem and have updated both the original articles and download examples.  Just to tidy things up – and so folks won’t have to wait next time, I thought I would describe how I got everything working again.  I’ll walk through the simplest example (the one from the “Consuming NDFD Web Services from a C# Client” post, but the debugging process and changes apply to all the examples that access the NDFD web service.

InvalidZipCode

 

1.  First I ran the original NdfdTest-Application example from the post.  Yes, things weren’t looking so good.

 

 

 

 

2. So I downloaded and built the original project – and there were errors:

badbuild

Clearly the service reference for the NDFD web service was not getting created.

 

3. Visual Studio can update service references if you ask so I right-clicked on the NdfdServiceReference and selected “Update Service Reference” from the shortcut menu and built the project again:

badcomplile2

Ah ha!  [Change 1] They changed the signature of the method used to get forecasts!  To see what’s happened, double click on the NdfdServiceReference from the Solution Explorer in Visual Studio then expand the NdfdTest.NdfdServiceReference node in the Object Browser and click on the ndfdXMLPortType interface.  It turns out they changed the signatures for all the methods, adding the ability to specify whether you want results in English (unitType.e) or Metric (unitType.m) units.  So I modified the NDFDgetByDay function, adding a unitType.e parameter and ran the application – and we are still getting an Invalid Zip Code error.  But now you can see in the output window we are getting a protocol exception:

exception

 

4. This is good information, but not enough.  So in Visual Studio I enabled breaking on any thrown Common Language Runtime Exception.  I ran the application again, entered a zip code and bang!

protocolexception

What the heck is going on?  We asked for for text/xml and we are getting text/html.

[Change 2] Somebody moved our web service – now it’s a web page instead!

webpage

 

5. I admit at this point, as soon as I updated the service reference, I should have gone to the app.config file and looked at what was generated for me.  Well, never too late…  Low and behold, there is a new address for the NDFD web service:

address

When I wrote this series of posts, the NDFD web service was at http://www.weather.gov.  Now it’s at http://graphical.weather.gov instead.

6. So finally, replace the address on our original ndfdXMLPort endpoint with the address from the newly generated ndfdXMLPort1 endpoint.  Everything is working again.

Getting the Current Weather Conditions

By , September 14, 2009

[January 2, 2012 Update: A number of changes have been made to the NDFD web service since I originally posted this article. Both the URL and the methods exposed by the service have changed. The text below and downloads have been updated to reflect these changes.]

Unlike weather forecasts which are available for any point in the US, current conditions are only available for specific spots where there are reporting stations. The good news is that there are more than 2000 reporting stations, so it’s not too hard to find one close to you. In this installment I am going to add the ability to get the current weather conditions to our library of weather tools. (If you are following along in this series, you will notice I have done a bit of mild refactoring, getting ready for the next steps – a local Windows “weather” service and a real desktop UI).

As usual, the folks at the National Weather Service have made our job pretty easy. You can take a look at what they’ve given us to work with here. Each observation stations get its own web page containing an Xml document (updated hourly) describing the current weather conditions. The URL for the page containing the current conditions for each weather station is formed by appending the station’s identification string to the base URL for current observations. So for example to get the current conditions for Smithfield, Rhode Island the URL would look like this:

http://www.weather.gov/xml/current_obs/KSFZ.xml

So now the puzzle comes down to how do we find the ID for the reporting station closest to the spot for which we want the current conditions? We’ve got a few choices, but all of them start with this document, the index of reporting stations. It contains an entry for every reporting station, formatted just like this excerpt for our Smithfield station:

<station>
  <station_id>KSFZ</station_id> 
  <state>RI</state> 
  <station_name>Smithfield</station_name> 
  <latitude>41.91</latitude> 
  <longitude>-71.4</longitude> 
  <html_url>http://weather.noaa.gov/weather/current/KSFZ.html</html_url> 
  <rss_url>http://weather.gov/xml/current_obs/KSFZ.rss</rss_url> 
  <xml_url>http://weather.gov/xml/current_obs/KSFZ.xml</xml_url> 
</station>
 

The key pieces here are the station_name, state and station_id. We’re also going to find the latitude and longitude useful a bit later. Since this is a reasonably large document (900kb) and we want to play really nice with the NWS folks so they’ll let us continue to use their toys (and we want our app to be responsive), we’ll download it once (using a BackgroundWorker) and cache a copy in IsolatedStorage (along with an option to refresh our cache when needed).

So armed with the list of all the reporting stations, there are a couple of routes we could take. On the one hand we could simply mimic what the NWS does on their current conditions web page:

  1. Show the user a list of all the states and ask them to pick one.
  2. Based on their selection, show them a list of all the observation stations in that state and ask them to pick one.
  3. From their selection, compose the URL for the current conditions page and submit a web request.

The TestFromStateStation sample application (available for download at the end of this post) implements this approach.

The other approach is to:

  1. Ask the user for the Zip Code for their location.
  2. Query the NDFD web service for a location (latitude, longitude) associated with the zip code.
  3. Compute the distance between the zip location and the location of each observation station in the reporting stations index (see above) to find the closest station.
  4. Use the station Id from the closest station to compose the URL for the current conditions page and submit a web request.

This technique relies on being able to determine the distance between two points on the earth’s surface. There are a number of techniques we can use – you can read a really good description of your options thanks to the folks at movable-type. For our purposes, the spherical law of cosines is good enough (see ObservationStations.GetDistance() in the NationalWeatherServiceLibrary project in the attached solution). The TestFromZip sample (see below) implements this approach.

So there you have it, the ability to get the current weather conditions using either state/station or zip code. As promised, here are the downloads: TestFromStateStation.exe, TestFromZip.exe and the complete VS2010 solution.

At this point we’re done with the web service pieces of the project; we know how to get forecasts and current conditions for anywhere in the US. Next it’s time to create the UI.

 

XmlTreeView – A Custom Painted TreeView

By , August 28, 2009

[January 2, 2012 Update: A number of changes have been made to the NDFD web service since I originally posted this article. Both the URL and the methods exposed by the service have changed. The text below and downloads have been updated to reflect these changes.]

Last time I described how to build a WCF client for the NDFD web service. The gist was we needed to create a custom TextMessageEncoder that could handle the ISO-8859-1 encoding used by the service. With that done, it’s time to start thinking about how to use all the data to which we now have access. My goal over the next week or so is to convert the NDFD client into its own local WCF service, hosted in a Windows service, to add the ability to get current weather conditions, and finally to build a couple of desktop clients (a desktop Gadget and an ActiveX control) that use the service.

But before I jump into that, I’m going to take a small detour and talk about custom painting a TreeView. This is motivated by the fact that the NDFD Xml data is a bit hard to visually parse when displayed as raw Xml, so I thought I’d simplify my debugging a bit by pushing the data into a custom painted TreeView.

There are a couple approaches to custom painting in a managed TreeView. The simplest (OwnerDraw) is to set the TreeView.DrawMode property like this:

this.DrawMode = TreeViewDrawMode.OwnerDrawText;

or

this.DrawMode = TreeViewDrawMode.OwnerDrawAll;

Then you override OnDrawNode and do your painting. The big benefit with this approach is that the TreeView does most of the heavy lifting – it determines when a TreeNode needs redrawn as well as the bounding rectangle – all you need to do is draw the contents of a node when requested.

The problem with this approach is that as soon as your node content gets a bit complex, you start down a slippery slope of increasing complexity as you add hit test code for selection highlighting etc. And if your drawing code gets the least bit complex, you will start to notice redrawing flicker as you resize windows containing your custom painted TreeView. (It’s unfortunate, but the built-in double buffering does not work for OwnerDrawText or OwnerDrawAll. You can get some improvement in the flicker by overriding WndProc and eating all the WM_ERASEBKGND messages, but it doesn’t really fix the problem.) So, except for the simplest situations, I usually end up using the second approach which is to do all the painting myself (UserPaint). It’s a little complex the first time, but most of the complexity is boilerplate code that you can reuse (the attached code is a pretty good starting point).

You enable user painting by setting several of the ControlStyles bits for your TreeView like this:

this.SetStyle(ControlStyles.UserPaint, true);
this.SetStyle(ControlStyles.AllPaintingInWmPaint, true);
this.SetStyle(ControlStyles.OptimizedDoubleBuffer, true);

Now only OnPaint and OnPaintBackground get called (OnDrawNode is out of the loop), and this is where the fun begins.

For starters, you’ll want to override both OnPaint and OnPaintBackground. For OnPaintBackground, I find it’s usually most efficient to do nothing more than insure the base class OnPaintBackground isn’t called:

protected override void OnPaintBackground(PaintEventArgs pevent)
{
	// We can do a better job of predicting where nodes will paint the background
	// and what area is left, so just disable background painting.
	//base.OnPaintBackground(pevent);
}

This means in OnPaint, we need to paint not only the TreeNodes but the empty area where there are no TreeNodes as well. So OnPaint looks something like this:

protected override void OnPaint(PaintEventArgs e)
{
	// If we have nodes, then we have to draw them - and that means everything
	// about the node.
	if (this.Nodes.Count > 0)
	{
		// If we have nodes then there has to be a Nodes[0]. Use it to get the
		// height of one TreeNode - in a TreeView, all nodes are the same height
		// (unless you are using PInvoke to call TVM_SETITEM and set TV_ITEM 
		// iIntegral member for a specific node - which we are not doing!).
		int nodeHeight = this.Nodes[0].Bounds.Height;
 
		int curY = 0;
 
		// Walk down the client area, stepping by the height of one row, looking 
		// for nodes.
		while (curY <= this.ClientRectangle.Height)
		{
			// This probably isn't the most efficient way to do this, but
			// at least we aren't writing our own code to walk the tree...
			TreeNode node = this.GetNodeAt(0, curY);
 
			// If this rect does contain a node, draw the node.
			if (node != null)
			{
				DrawTreeNodeEventArgs args = new DrawTreeNodeEventArgs(
					e.Graphics,
					node,
					e.ClipRectangle,
					GetNodeStates(node)); // manufacture the proper TreeNodeStates
 
				DrawCustomNode(ref args);
			}
 
			// If this rect doesn't contain a node, we need to fill it with the 
			// background color.
			else
				e.Graphics.FillRectangle(
					_backBrush, 
					0, 
					curY, 
					this.ClientRectangle.Width, 
					nodeHeight);
 
			curY += nodeHeight;
		}
	}
 
	// If there are no nodes in the tree, we're going to have to paint the 
	// entire background.
	else
		e.Graphics.FillRectangle(_backBrush, this.ClientRectangle);
 
	// Don't need - in this case we can guarantee no one is listening...
	//base.OnPaint(e);
}

As you can see, our job is to just walk down the client area of the TreeView, one node height at a time, checking to see if there is a node there. (There is a lot of optimizing opportunity here.) If we have a node we draw it; if there isn’t, we draw background. That’s all there is to it – well almost. You probably noticed there are a couple of methods stuck in the middle of OnPaint that we haven’t talked about yet – and these can be a bit of a pickle.

The first problem we need to resolve is addressed by the GetNodeStates method which supplies the TreeNodeStates parameter for the DrawTreeNodeEventArgs constructor. The stock TreeView that you are familiar with has a curious behavior when it comes to highlighting selected TreeNodes. If you click on one node to select it, its background gets painted with SystemBrushes.Highlight and all is good. But if now you click-and-hold on a second node you will notice the first node loses its highlight and the second node is now highlighted – but it gets even more complex. If now (while still holding your mouse button down) you move your mouse a bit (say more than 15 pixels or so) the highlight reverts to the first node. This behavior is made possible by TreeNodeStates.Focused. It turns out that in a TreeView, node highlighting is indicated by Focused rather than Selected, so it’s the coming and going of Focused for the first and second nodes that is prompting the highlighting behavior we just saw. The problem is that we don’t actually have a way to directly test for the Focused state. If you are using OwnerDrawText or OwnerDrawAll you get the Focused state handed to you when your OnDrawNode override is called, but when doing our own drawing in OnPaint, we need to manufacture the Focused state and this is the purpose for GetNodeStates (this is the secret sauce):

private TreeNodeStates GetNodeStates(TreeNode node)
{
	TreeNodeStates states = 0;
 
	// If mouse button is not down, then normal rules apply - if a node is
	// selected, then it also has focus. We set _mouseDownNode in our
	// OnMouseDown override and clear it in our OnMouseUp override.
	if (_mouseDownNode == null)
	{
		if (node.IsSelected)
			states = TreeNodeStates.Selected | TreeNodeStates.Focused;
	}
 
	// But if a mouse button has been clicked on a node, then we need to check
	// to see if that node is currently drop-highlighted.
	else if (node != _mouseDownNode)
	{
		// If this is the currently selected node, but a mouse button has been
		// clicked-and-held on another node and that node is currently showing 
		// the drop-highlight, then we don't show selection highlighting for this
		// node.
 
		if (!IsDropHighlighted(_mouseDownNode) && node.IsSelected)
			states = TreeNodeStates.Selected | TreeNodeStates.Focused;
	}
 
	// Othewise if this is the node that has been clicked-and-held and if
	// this node is currently showing drop-highlighting, then show it. We 
	// still need to check to see if drop-highlighting is being shown for 
	// this node because if the user clicked-and-held on this node, but
	// then started to move the mouse, drop-highlighting is removed by
	// the underlying Win32 tree control.
	else if (node == _mouseDownNode)
	{
		if (IsDropHighlighted(node))
			states = TreeNodeStates.Focused;
	}
 
	return states;
}

The final pieces here are a couple of calls to the underlying Win32 tree control to get the state of the nodes in question from which we can determine if a particular node is drop-highlighted:

private bool IsDropHighlighted(TreeNode node)
{
	const int MASK = NativeMethods.TVIF_STATE | NativeMethods.TVIF_HANDLE;
 
	// TV_ITEM stateMask doesn't seem to matter when reading state - we get
	// all the state bits anyway (Windows7).
	const int STATEMASK = NativeMethods.TVIS_DROPHILITED;
 
	NativeMethods.TV_ITEM lParam = 
		new NativeMethods.TV_ITEM { 
			hItem = node.Handle, 
			mask = MASK, 
			stateMask = STATEMASK };
 
	UnsafeNativeMethods.SendMessage(
		new HandleRef(this, this.Handle), 
		NativeMethods.TVM_GETITEMW, 
		0, 
		ref lParam);
 
	return (lParam.state & NativeMethods.TVIS_DROPHILITED) != 0;
}

(The definitions for the various Win32 methods, structures and constants can be found in the attached sample.)

The second method we haven’t talked about is DrawCustomNode – this is where I actually get around to drawing the TreeNodes. Since my goal for this project was to create a simple view of an XmlDocument, the code just implements the stylized drawing to do that – and this is where you would put your code to draw a BarGraphTreeNode or whatever. In my case, since I am “syntax coloring” Xml, there is quite a bit going on to insure various segments of the colored text line up correctly – but that’s another story.

The attached sample (executable, source) contains a fully working test app using the NDFD web service to get weather forecasts for specific ZIP codes. It pushes the Xml forecast data into our new XmlTreeView. I hope you enjoy.

Panorama Theme by Themocracy