Recent Changes - Search:


Code:


Social:


My journals will take the place of a blog. If you go to the Journal page there is an RSS feed to subscribe with.

(:blogcal group=Journal:)


Sites I take responsibility for

West Volusia, Florida

Documents

Technical

General

Me

Living

Places I frequent

Hobbies

Humor

Items for sale:


Clipboard

edit SideBar

NOAAPORT

Include our styles below Infobox - invoke as >>infobox<< ... >><<

Codebox: - invoke as >>codebox<< ... >><<

warnbox: - invoke as >>codebox<< ... >><<

editingbox: - invoke as >>codebox<< ... >><<

noticebox: - invoke as >>codebox<< ... >><<

Page bread crumbs: Main - Weather - NOAAPORT

Pages by tags: (:listtags:)
Subscribe to this wiki: RSS Feed RSS or subscribe to this page for changes: RSS Feed RSS
496 articles have been published so far. Recent changes
(:addThis btn="custom":)

2016-03-22: This site is being moved to my main site at https://kevininscoe.com/wiki as part of a consolidation to one domain.


Notes

http://www.weather.gov/om/marine/noaaport.htm

"The NOAAPORT broadcast system provides a one-way broadcast communication of NOAA environmental data and information in near-real time to NOAA and external users. This broadcast is implemented by a commercial provider of satellite communications utilizing the C-band. It's primary purpose is for providing internal communications within the National Weather Service and for providing forecasts, warnings and other products to the mass media (newspapers, radio stations, TV, etc.), emergency management agencies, and private weather services.

The NOAAPORT satellite communications system is operated by GTE Corp., under contract to the NWS. The system uses satellite transmitting (i.e. "uplink") equipment at NWS forecast offices throughout the continental U.S., Alaska, Hawaii and Puerto Rico. Each uplink site transmits NWS-generated weather information products which are then re-broadcasted via satellite to users."

Internet stream

Click on the above link for more information.

Some of the reasons for having your own Noaaport ground receiving station are:

  1. Not having to depend on the Internet to receive weather information from the National Weather Service.
  2. The information will be received much faster and more reliable.
  3. Ability to receive high quality weather satellite (GOES) at rates and speeds not normally available by Internet.
  4. Ability to receive all of the Nexrad Radar Level3 products at faster rates.
  5. Basically any product that is generated by NOAA/NWS is on Noaaport

More links:

http://www.weather.gov/noaaport/html/noaaport.shtml

http://www.ssec.wisc.edu/mcidas/software/xcd/dvb/

http://www.wxmesg.com/NoaaPort.htm

http://groups.yahoo.com/group/noaaport/

http://weather.unisys.com/noaaport/index.html

http://www.nws.noaa.gov/noaaport/html/channels.shtml

http://www.hwic.net/projects/weather/noaaport/twinhan_1020/

http://wxmesg.blogspot.com/2008/01/noaaport-installation.html

http://weather.unisys.com/noaaport/text_summary.html

http://noaaport.poes-weather.com:8081/

http://directreadout.noaa.gov/Miami04/program.htm

http://www.unidata.ucar.edu/cgi-bin/rtstats/iddstats_vol_nc?NEXRAD3+noaaport.admin.niu.edu

http://www.unidata.ucar.edu/

http://www.poes-weather.com/

http://www.geo-web.org.uk/

http://tech.groups.yahoo.com/group/weather-satellite-reports/

http://wxmesg.blogspot.com/2008/01/noaaport-installation.html


Software:

http://www.noaaport.net/

http://www.unidata.ucar.edu/software/gempak/

http://www.unidata.ucar.edu/software/ldm/

http://serv1.k9nl.net/noaaport/

http://tech.groups.yahoo.com/group/APTDecoder/

http://www.poes-weather.com/

http://noaaport.poes-weather.com:8081/


Hardware:

http://www.novra.com/s75.htm

Notes seen on mailing lists (quoted):

> I am about to buy a satellite dish used. Believe it was used for TV reception before mini dishes took off and cable wasn't available. So its not new, but should function.

If the dish is in good condition, the worst thing you'll need to buy for it is a decent LNB.

> I probably will not install it until the Spring. In the meantime, I'll try to start acquiring everything else I need to ingest the data into a PC in my office.

If you've got a PC, you're half way there. All you need now is a DVB data receiver of some sort. Either the Novra S75s people here like to use or a datacapable DVB PCI card like the Twinhan or Broadlogic.

> Anyway, just thought I'd post something to say say Hello again. Any advice or pointers are welcome. I now live in SE Michigan and my the front of my home directly faces South and the back of it faces North (not true North of course). Advice on dish placement, etc. would be welcomed.

From where you're at, you'll need at least a 10ft dish and it will need to be mounted somewhere where it can see the southwestern sky. You'll be looking at AMC2/4 at 101 degrees west.

I am able to receive good NOAAPORT data using one Windows computer to both control the Twinhan receiver and also decode the signal.

 So, if you are enjoying NOAAPORT as a hobby and prefer to keep the cost

low, you need only a dish with the LNB feed and a Windows computer with a PCI TV."

"From what I know, the Novra S200 will be able to receive and pass NOAAPort data just as the S75+ (and S75), and it is dvb2 capable also."

"And there are some quite nice cheap cards and USB dongles that demod DVB-S2 very well also. Somewhat pricier than the cheapest Twinhan or Broadcom but not budget busting...

Only issue is the DVB-S2 signals are often 8PSK and that implies better signal quality (bigger dish or strong tranmission from the satellite).

A switch to DVB-S2 would likely render marginal dishes more marginal or even completely useless..."

Dish:

"AMC-4 died. Americom brought AMC-2 out of storage and moved it over, then turned the lights out on AMC-4. Then they renamed AMC-2 to AMC-4.

If you use an older sat meter that locks on analog, it comes up as AMC-2. If you use a newer meter that locks on digital, it comes up as AMC-4, as Americom reprogrammed it to display as AMC-4."

http://www.ses-worldskies.com/worldskies/satellites/01_amc-fleet/amc-4/index.php

http://www.ses-worldskies.com/worldskies/satellites/01_amc-fleet/index.php

Unidata, IDD and history:

"The Unidata Internet Data Distribution (IDD) System now delivers real-time atmospheric science data to nearly 100 universities nationwide. The IDD is a fully distributed system based on Unidata Local Data Manger (LDM) systems running at each of the participating sites. In the IDD, the LDM at a source site takes data from the source and sends it to LDMs at downstream relay nodes who in turn relay the data to other sites. This fan-out approach scales with an increasing number of sites as long as enough relay nodes can be established. The IDD/LDM technology has been shown to work in the real world of the Internet for a substantial, but limited, amount of data."

http://sysu1.wsicorp.com/unidata/intro.html

http://www.unidata.ucar.edu/

http://www.ucar.edu/

http://www.ncar.ucar.edu/

http://www.unidata.ucar.edu/support/help/

http://www.unidata.ucar.edu/software/idd/vbns.html

http://www.ssec.wisc.edu/mcidas/software/xcd/dvb/

Politics:

<quoted from internet sources>

"This data is already accessible on the internet via IDD, but it is not publicly available as a real-time feed. However, there are several repositories where this data can be accessed via HTTP or FTP in a non-real-time manner."

"NWS has access to most of the data via IDD/LDM if needed."

"NOAAPORT via IDD/LDM if I have understood it correctly is only avaiable for university users such as students or dot edu folks? I did try install the core LDM and got stuck due to I did not have enough credit (not a university member or not having a dot edu email)

I am signed up at their email list and it is very busy and most interesting to follow their progress. http://www.unidata.ucar.edu/"

"Need more than a .edu email address, but correct."

"The IDD distribution "rule" has very little to do with bandwidth capacity; it is about IP (Intellectual Property), contractual obligations to data providers, and support for that IP and data connectivity.

You are mixing apples with oranges . . . the IDD and NOAAPort. The IDD was established for the University and research community (including

  • .gov) and includes proprietary data IP - thus the restrictions on who

gets to attach to the IDD. It hasn't been recently, but I am aware of at least one instance when a University node was nearly dropped because it allowed access to its node from a commercial entity without prior consent from UNIDATA. If you are an administrator or faculty member at a University, you can do what you want . . . but understand that violating the rules could get you cut off from the IDD.

And there are "rules" to NOAAPort, as well - WMO Resolution 40. I have been relatively silent about the redistribution efforts of some on this list for NOAAPort data . . . however, that effort can screw us all in the end if the source nation for protected data - data currently on NOAAPort - feels that the US (via NOAA) is not doing enough to abide by WMO Resolution 40, they will simply yank it or rewrite the contract with NOAA so that their data no longer on NOAAPort. In fact, this has already started to take place with data from Bracknell - NOAA has not been able to put new ECMWF or other UK data sets on NOAAPort because of the relative lack in IP protection as perceived by the Brits.

The thing with NOAAPort is that it is relatively assured that reception will be in the western Hemisphere by the footprint of AMC-2/4. Unless R40 has changed in the past year, there are no R40 limitations on data distributed or collected in the western Hemisphere except lightning data and ACARS - and it is that reason that lightning data and ACARS are encrypted on NOAAPort. It is the redistribution of data from a source nation *back* to the source nation or compact nation that violates R40.

The fact is, what Jerry and Patrik are doing are exactly what several European countries do NOT want to happen; data collected at a ground station in the US distributed, with no value add or limitation, back to an EU nation that may or may not have IP limitations.

And just because you are not "selling" that data does not make you immune to a tort for damages; I guarantee that if the Republic of Ireland wants to make an example of Jimmy Trailercourt and his backyard "TeeVee" dish allowing access to protected Irish buoy data - Jimmy will be out of a trailer. Or worse (at least for all that benefit from NOAAPort) Ireland will simply no longer give NOAA the ability to redistribute said data except on the AWIPS LAN.

I'm all for free data . . . but ignorance of the rules is not a defense; abide by the UN/WMO resolutions or suffer the fate of even additional loss of international data on NOAAPort. And it is not a stretch, as NOAA continues to add data to the AWIPS LAN because it cannot put it on NOAAPort because a bunch of yahoos don't like rules - that NOAA will discover that it can distribute all its data needs via the AWIPS LAN, and then bye-bye NOAAPort.

In the end, NOAAPort's purpose is for feeding WMOs and other government centers. They could care less, contractually, if Gilbert receives data in Illinois (not picking on you Gilbert - you are one that I know understands) - a non-*.gov can call NCF all they want, but if the NOAA facilities are not having a problem, they could care less what issues non-*.gov users may have. And Americans on this list can bemoan the "I'm a tax payer" argument, but fact is that being a tax payer doesn't entitle you to raw data - at least from NOAA in real-time. It entitles you to information that helps you protect your life and property - and NOAA's stance will be that the NWR accomplishes that, and the private media fills in the gaps. And Patrik or David T. certainly does not pay taxes in the US."

"I'll chime in...

1) BANDWIDTH - There is no restriction that I know of to use LDM to redistribute data... weather or otherwise. The only real restriction is bandwidth. Since the LDM does not use multicast (unless I've missed something), it can be costly to redistribute a lot of weather data over an open Internet connection.

The full NOAAPORT feed is about 4-8mbit/sec. This means a T1 connection is not enough to get the full feed. If you are relaying, you need to multiply the incoming feed by the number of outgoing feeds you have.

Most small companies generally have limited bandwidth and thus probably couldn't handle an LDM feed with any significant about of weather data on it.

2) RESTRICTED DATA - The various feeds of weather data do have some restricted, proprietary and commercial data on it.

First you have prorietary data. For example, NOAAPORT has NLDN lightning data which is reserved for government use only. We've been told that our NOAAPORT customers cannot use the NLDN data on NOAAPORT even if they have a license to read NLDN formats.

Second, there are restricted data such as Resolution 40 data from other countries. NOAAPORT contains a lot of international data that is under Res. 40. This prevent redistribution of data back into the countries or regions that create the data. This is because, outside of the US, weather data and information are not subsidized by the Federal Government and thus they rely on sales of that data to recover costs. Models like the ECMWF and UKMET models that are on NOAAPORT can have limited redistribution. This is a fuzzy gray area. In my talks with the NWS, they would prefer people not to redistribute those models even though it's OK to redistribute them within the US. But the NWS realizes that once you start open redistribution, people exploit the privilege and violate redistribution rules. The NWS can lose their rights to put that data on NOAAPORT if we abuse the privilege.

If you look at Unidata's IDD, there are a lot of data that are either proprietary or restricted. Clearly redistributing that feed outside of the educational arena would not be allowed.

3) CHANGES TO NOAAPORT? - I heard some talk about changes to NOAAPORT in the future. This is because of changes with the AWIPS system and a reworking or how data might be transmitted. Currently there is a lot of data duplication on NOAAPORT. A lot of this is due to the requirements of the current AWIPS system. But this will change with AWIPS II. The future model for NWS data distribution could change as a result:

I) NOAAPORT II - this is a satellite broadcast of data but only data required by all WFOs. Think of this as the lowest common denominator.

II) LDM - this is an internet broadcast of data for that information that is required by each WFO or what I call locally required data.

III) Web services - these are one time requests of data for special situations.

There is no reason why this couldn't be done on a wider scale. Most weather enthusiasts don't need a live feed of weather data and it's possible that web services and limited LDM data distribution could be possible and not overwhelm local bandwidth restrictions. I haven't really looked into what web services are available for weather data or whether this is even being approached outside of the NWS. But these are possibilities in the future."

NOAAPORT is not Unidata:

"I was part of the original Unidata steering committte in the early 80's and created the specifications for the minimal workstation. You are correct in that the Unidata stream is limited to Unidata membersm which are primarily universities. There is some very limited participation on the part of companies who are involved with the NWS/NOAAport project. No the Unidata datastrem is not the same as the NOAAport data stream. It is a far better datastream since it contains a lot things NOAAport does not, think CRAFT and CASA."

"To add to the discussion of restricted distribution, if I understand it right, there is also the ECMWF & UKMET full model data on the LDM/IDD broadcasts which are limited to .gov & .edu domains. They want what they perceive as their competition ( Jo Public & his private meteorological forecasting / value adding company) to pay for that data & the only European data on NOAAPort is the limited data cleared for public distribution. To get the full European GRIB data (and some obs data as well, I think) you have to sign a contract with the ECMWF declaring you are a .gov or .edu or I think, a private individual who will not redistribute the full data...otherwise you have to pay for it...just like the full Meteosat data. The METSAT is encrypted and you have to pay for a decryption unit to get everything which is not transmitted in the 'clear' which is every three hours."

"EUMETSAT data is free to private individuals, schools colleges, and many research organisations. The expensive decryption unit for 1.7GHz systems was phased out some years back when Meteosat-8 and Meteosat-9 took over the Europe/Africa mission, and EUMETCast replaced direct HRI broadcasting. You are correct that people who make a profit from the data, or redistribute it (TV stations for example) may have to pay.

Strangely, Meteosat-8/9 "in the clear" data is only every six hours, whereas everyone else's is every three hours. Erm?

EUMETSAT also offer data over the Internet, which is free if you accept a 24-hour delay."

Changes proposed for NOAAPORT: http://preview.tinyurl.com/ygabw64

"I was at that Partners Meeting back in June. The bottom line is that there won't be any major changes to NOAAPORT before 2013 but there could be significant changes after that. They mentioned that there are contractual obligations to keep the current content as is for a while. The question of the move to GRIB2 was asked and that transition for NOAAPORT is still on hold.

Right now, the easiest way to redistribute model data is over the SBN and the NWS is planning on ramping up the amount of model data. Also, there are some new radar products on the way and some additional satellite imagery down the line. So they are investigating a move to DVB2 which has better compression and error recovery. I don't know how this will affect the receivers but this switch could come as soon as next year."

"Some in the private meteorological industry have argued that the NWS only have two functions:

1) collect weather data 2) issue public weather warnings

I haven't heard anyone state they want to deal with the infrastructure of collecting the data but they don't want the NWS to redistribute the data. In other words, NWS data is for NWS use only. This was actually one of the first edicts of the Reagan administration. Prior to 1981, if you wanted weather data, you could install a telephone line to a local NWS forecast office and they would give you a variety of feeds: DDS, PPS, FAA604, etc. After 1985, people, companies, and universities had to get their data through a private company. Those were WSI, Kavouras and Alden (Unisys was added in the early 90s). This cutoff of data access gave rise to entities such as Unidata which lobbied for cheaper access to universities and other educational institutions. Unidata ended up contracting with Alden to get the data and developed the LDM to save the data. This also gave rise to the era of satellite distribution of weather data as these private companies needed a way to broadcast data and landlines wouldn't work.

So for 13 years, if you wanted weather data, you had to get it through a private company. That all changed with two new concepts. First was the internet. Web sites that went online in the mid 90s were becoming a way for people to once again gain access to free weather data. Second, was NOAAPORT. In one fell swoop, NOAAPORT killed off the private weather data distribution system. There are smaller Ku band satellite feeds still available but it's small business relative to what it was in the mid 90s.

There are many in the private industry who want a return to the days of the 90s where they were the access point of data outside of the NWS. But I believe this is unrealistic now. The reason is the amount of data that is out there, which is doubling about every 5 years. It has now grown past the ability of private industry to replicate. Can you imagine a WSI or Accuweather launching their own NOAAPORT service?

The question now is how much data is too much. The NWS argument all along has been that if they create the data for internal use, they might as well put it up on NOAAPORT or onto the FTP and web servers for others to use. The private industry argument is that the NWS should not provide data that is already being provided in the private sector. In other words, the NWS should not infringe on existing private sector business. Well, the NWS has repeatedly violated this argument in the last decade. I hear this over and over again in the NWS Partners Meetings which at times are all-out verbal feuds. Private industry feels that if they can't talk the NWS out of invading their business space, they'll go to Congress to get it done that way.

The second issue is public weather warnings. Private industry does not want to take this on because of liability issues. The government has a blanket immunity from prosecution. Even though some think their teams of lawyers can get around this, almost everyone is in agreement that warnings should be left to the NWS. But other than warnings, the private sector believes they should be able to issue their own watches and advisories.

So in the battle between private and public weather information, we seen various pushes one way or the other over the last 30 years. The Santorum bill would have virtually eliminated the weather service overnight relegating it to the role I described above. It's interesting because over the last decade, public access to the NWS data and forecasts has been dwindling. Go back to the mid 90s and watch a TV broadcast. Most of them reported the NWS forecasts for the area. Today none do. All weather forecasts are now filtered through a private company that provides the data and forecasts to a TV or radio station. Most companies run their own computer models and create their own watches and warnings even though the latter is for internal use only. Unless you go to weather.gov or a university web site, you are not getting NWS forecasts. So to a certain extent, much of what Santorum argued for is already in place. But what that bill would have done was to eliminate the NWS from creating those forecasts, watches and advisories entirely. They're not widely used anymore anyway is how one person described it to me.

The new argument I've heard is that with the newer technology, the NWS can do more with less. As a result, the NWS is free to now venture into areas that private industry did ten years ago. In other words, a duplication of effort. So why not cut the NWS budget. Of course, with any government entity, they meet budget cuts with fierce opposition. We've seen this in the past. In the early 90s, the Bush administration wanted to cut the NWS budget by about 15%. The NWS responded by saying it would cut back on data acquisition by reducing the surface observation network by 30% and go to once a day soundings. They also said the Hurricane Center hurricane hunter program would be all but stopped. So to fight the cuts, they would drop essential programs and not any fat within the agency. The budget cuts were dropped.

It seems like every 3-5 years we see another one of these proposals to cut NWS funding. Now we see that Senator Hutchinson wants to transfer about $150 million out of the NOAA budget (about an 8% cut) and move it into the new efforts of immigration detention. If you're not familiar with that, it is an effort to create holding camps for illegal immigrants where they will be held awaiting deportation. Texas and California are to have the largest networks of these camp. Hence the effort by the Texas Senator to transfer funding.

We'll see where this goes!"

"This brings up an interesting point. The use of data....

One issue private industry has with the NWS is not the availability of data but how it's delivered. Clearly the mission of the NWS is different than that of private industry. Here are a couple of examples:

1) live radar data 2) 5 minute or better surface observations 3) 15 or 30 minute model output 4) southern hemisphere anything

When private industry distributed NIDS, many of the requirements was to have data sent from the radar to the end user in seconds. But when the NWS took over NIDS distribution, this was no longer possible. Even though it was better that the NWS eliminate the duplication of effort (i.e. every NIDS vendor having to have 140 telephone cricuits, one for each radar), it didn't help that the latency in the NWS system was almost a minute. This is because the NWS never had a critical need for real time radar data outside of the radar site at each WFO. Industry suffered as a result.

Also, the TV industry needs live radar data. The problem on the NWS side has always been that those radars do volume scans and thus you get a tilt 1 reflectivity product once every 5-10 minutes, not continuous like TV stations would like. So TV stations went out an invested in their own radars. There are actually far more of those radars out there than NWS WSR88Ds.

Industry would like to see continuous updates of surface observations, not the 1 hour or at best 20 minute updates. Again, this is outside of the needs of the NWS. Now, you can get this from networks like what WeatherBug has put together. You could argue this network is far more complete than the NWS surface network.

Industry has been demanding better model data. A key need of private industry is high temporal resolution output, say 15 minute output. The NWS only does 1 hour at best and the better models are sent at 3 hour intervals. This has sent many in private industry to create their own computer model runs (either with MMM or WRF) to get the output they need.

I know in energy trading and commodity futures, need for international data is critical and there is a real lack of high resolution model data outside of North America. Again, this is not a critical need of the NWS.

So what does private industry do?? I think the frustration is not whether private industry is more cost effective in collecting the data. I think it's more related to the fact the needs of private industry are totally different than the needs of the NWS. I think private industry controlling the observation network comes more from a need to control the output than doing it cheaper.

Clearly competition works but replicating huge networks of observing platforms and the infrastructure to collect, manage and distribute it doesn't make any sense. If you had it over to one company with exclusive rights to collect and distribute means that access to that data could become very costly. A good example of that is lightning data.

There is no doubt the NWS, which lacks competition, has become complacent when it comes to these issues. But to replace the NWS with private collection and to make it competitive doesn't make sense either. How you correct it is an interesting debate."


Attachments:

Attach:NOAAPORT_Presentation_Miami_2004.pdf

Attach:The_Rise_and_Fall_of_Amateur_Direct_Reception_of_Weather_Satellites.pdf


Kevin's Public Wiki maintained and created by Kevin P. Inscoe is licensed under a
Creative Commons Attribution 3.0 United States License.

Back to my web site - http://kevininscoe.com

Edit - History - Print - Recent Changes - Search
Page last modified on December 23, 2009, at 01:38 PM EST