Talk:Main Page

From CHG-Wiki
Revision as of 02:19, 25 September 2013 by Tamuka (talk | contribs)
Jump to navigationJump to search

Team Discussion/Tracking on GeoCLIM development

Data for GeoCLIM

Comment on Data for GeoCLIM


Latest GeoCLIM download

  • GeoCLIM - updated 20 Sep 2013[1]
  • CHIRPS archive - 0.2 degrees; GeoCLIM-compatible [2]

Latest features/updates

  • GeoCLIM can now take in datasets from several different file-naming conventions and any folder on the computer. This should mainly be done under the setup tool, but is also accessible from the File menu
  • GeoCLIM can now handle rainfall, temperature (avg, max and min) and PET analysis in daily, pentadal, dekadal and monthly formats. I have only tested rainfall dekadal and monthlies
  • All references to unbiasing have been removed
  • All references to RFE have been made either (a) more generic for climate surfaces in general (b) refer explicitly to CHIRP or
  • IRE has been renamed to BASIICS: - Background-Assisted Station Interpolation for Improved Climate Surfaces: to reflect usage with other datasets such as temperature, and to reflect an improvement over the traditional distance-only-based interpolation techniques
  • File-new-dataset option is now linked to an appropriate function


Decisions

  • Windows XP compatibility will be maintained


To-dos

Action: Diego

  • send Tamuka some (any) max and/or min temperature station data to allow testing of the BASIICS routine for temperature. Alternatively you can test the routine from there and provide feedback Tamuka100 03:19, 25 September 2013 (PDT)
  • COMPLETED (2013/09/25): compile, zip up and send to Tamuka shapefiles of CILLS, GHA, and the five countries in the EAC: Kenya, Uganda, Burundi, Rwanda and Tanzania. shapefiles shold be in 1st, 2nd and 3rd level admin boundaries where possible
  • COMPLETED (2013/09/25): compile, zip up, and send to Tamuka tmax and tmin climatologies.

Action: Tamuka

Action: Chris

  • provide guidance on bulletproofing and on download CHIRPS archive button. Tamuka100 03:19, 25 September 2013 (PDT)


Questions/clarifications/comments/suggestions

  1. I think we should also generate a sample folder with various small datasets as part of a training kit for participants. Tamuka100 03:19, 25 September 2013 (PDT)
  2. Chris, what do you mean by "we should try to bullet proof GHA-monthly rainfall time series and analysis"
  3. Now that I have had a look at the data in the ftp folder, I dont think there is a need to add a button to download CHIRPS archives, since all the data is already available for download one at a time, which is actually easier esp for users with slow data connections (incl some of those in Africa). Alternatively, we can have the "download archives" button download the archive to some folder specified by the user, then he/she can copy the arcgives to other computers, or ingest it to GeoCLIM at a later date? What do you think Chris?
  4. If you do want to create CHIRPS archive files (which will be a good idea for when you are distributing the program), the archive is created as follows:


How to make a GeoCLIM archive

  • the CHIRPS (or other) data should be put in a single directory with a descriptive name (e.g. "CHIRPS_Africa_Monthly"),
  • a text file with the filename "_dataset" (no extension) should be created in the same directory. The text in the "_dataset" file should be as follows:
BEGIN DATASET
 DATASETNAME CHIRPS_Monthly
 DATATYPE ppt
 DATAPREFIX chirps
 DATADATEFORMAT YYYYMM
 DATASUFFIX .bil
 DATAMISSINGFROM -99
 DATAMISSINGTO -99
END DATASET
  • Make the appropriate adjustments to the _dataset file so that it matches the actual situation in terms of names etc
  • The directory with data should then be zipped up, and the .zip extension should be changed to .climdata
  • In the above example, the files would be called e.g. chirps201304.bil, which would be the chirps file for April 2013(2013-04).
  • There are 5 data types with different codes (in brackets below), namely:
    • precipitation (ppt), average temperature (tav), maximum temperature (tmx), minimum temperature (tmn) and potential evapotranspiration (pet)


DATADATEFORMAT codes for _dataset file

Below is a detailed list of codes for datadateformats which should be used for the different filenaming conventions in the _dataset file for the archives:

Days

  • YYYYMMDD: 4-digit year; 2-digit month; 2-digit day
  • YYYYMMD: 4-digit year; 2-digit month; 1-digit day
  • YYYYJJJ: 4-digit year; 3-digit Julian day (001-366)
  • YYYYJ : 4-digit year; 1-to-3-digit Julian day (1-366)
  • YYMMDD : 2-digit year; 2-digit month; 2-digit day
  • YYMMD : 2-digit year; 2-digit month; 1-digit day
  • YYJJJ : 2-digit year; 3-digit Julian day (001-366)
  • YYJ : 2-digit year; 1-to-3-digit Julian day (1-366)
  • MMDD : 2-digit month; 2-digit day [Averages]
  • MMD : 2-digit month; 1-digit day [Averages]
  • JJJ : 3-digit Julian day (001-366) [Averages]
  • J : 1-to-3-digit Julian day (1-366) [Averages]

Pentads

  • YYYYMMP: 4-digit year; 2-digit month; 1-digit pentad
  • YYYYPT : 4-digit year; 2-digit pentad (01-72)
  • YYMMP : 2-digit year; 2-digit month; 1-digit pentad
  • YYPT : 2-digit year; 2-digit pentad (01-72)
  • MMP : 2-digit month; 1-digit pentad [Averages]
  • PT : 2-digit pentad (01-72) [Averages]
  • P : 1-or-2-digit pentad (1-72) [Averages]

Dekads

  • YYYYMMK: 4-digit year; 2-digit month; 1-digit dekad
  • YYYYEK : 4-digit year; 2-digit dekad (01-36)
  • YYMMK : 2-digit year; 2-digit month; 1-digit dekad
  • YYEK : 2-digit year; 2-digit dekad (01-36)
  • MMK : 2-digit month; 1-digit dekad [Averages]
  • EK : 2-digit dekad (01-36) [Averages]
  • K : 1-or-2-digit dekad (1-36) [Averages]

Months

  • YYYYMM: 4-digit year; 2-digit month
  • YYYYM : 4-digit year; 1-digit month
  • YYMM : 2-digit year; 2-digit month
  • YYM : 2-digit year; 1-digit dekad
  • MM : 2-digit month [Averages]
  • M : 1-digit month [Averages]




Historical email discussions below:

Tamuka Here are a few comments - I like the logo for the new GeoCLIM - In the file-new-dataset definition, does not go anywhere - Take out unbiased RFE when dowload - Please take the unbiasing rainfall estimates - Take out the install rainfall unbiasing files - Overall the program freezes

Thanks Diego. Noted. Please see a few more changes as already discussed by email:

- GeoCLIM can now take in datasets from several different file-naming conventions and any folder on the computer. This should mainly be done under the setup tool, but is also accessible from the File menu - GeoCLIM can now handle rainfall, temperature (avg, max and min) and PET analysis in daily, pentadal, dekadal and monthly formats. I have only tested rainfall dekadal and monthlies - I have taken out all references to unbiasing - I have made any RFE references more generic - to either refer explicitly to CHIRP or to climate surfaces in general. I was a bit stumped with the IRE though - should I rename IRE to ICE (improved CHIRPS interpolation)? I finally decided that since we would want to implement this procedure with any climate dataset, a complete name overhaul was in order. So I propose: BAIISICS (pronounced basics) - Background-Assisted IDW Interpolation of Stations for Improved Climate Surfaces. I think this new name also emphasizes the point that we are not just improving the rainfall grids, but we are also saying this is a better product than you would get by just doing a plain-old distance-based interpolation. What do you think, can we adopt this new name?


A couple of questions regarding the downloading: - please send me an operational ftp site and folder and naming convention where the CHIRPS is operationally updated to so that I can hardcode this. Once I have this, I will change the default download from RFE to CHIRPS. - The CHIRPS archives (which you want us to have a download utility for) will need to be in a specific format so that they can be ingested into the GeoCLIM (a zipped file with a subfolder where all the data are, and a text file called "_dataset" which contains information on the type of file in the archive. The zip file should be renamed to give it a .climdata extension. I would have to work with Pete or whoever is in charge to tell them the exact details of this file. In the meantime, I have put some dummy forms. When a user does a clean install (uninstall GeoCLIM, then delete the My Documents\GeoCLIM folder), the first time the user runs GeoCLIM, they will be asked if they want to (a) download CHIRPS, (b) import some existing data archive or (c) Cancel.

One comment regarding countries: - GeoCLIM is more likely to be implemented successfully in a country where the files for for that country are already available by default in the country. I say this from experience with GeoWRSI - making a region from scratch is a laborious process, and it's much better if the user can just click and choose. As such, please send me a list of countries (and if possible, public-domain province-and-boundary-level administrative units shapefiles for those countries) where you plan on providing training


One comment regarding the BAIISICS: I think it would be great if we could put a utility in the program which would allow users to create an archive of the BAIISICS outputs, which archive can be easily transferred to and imported by another computer. To do this however, the best would be to use a function in the .Net Framework 4.5 which would allow us to easily zip up files. Unfortunately, .Net Framework 4.5 is not supported by Windows XP, which means if we did that, GeoCLIM would not run on Windows XP. Unfortunately, I have recently come across many computers in some govt departments still running windows XP. So how should we proceed with this?

While you think about this, I leave you to enjoy the latest greatest GeoCLIM (it would be the greatest so far I guess, since, as Diego observed, the previous one was freezing a whole lot - let's hope for slightly sunnier weather for this version!)

Link to latest GeoCLIM:

https://docs.google.com/file/d/0B_H-a39WNYAUUjhwRXYwMkJIZ0U/edit?usp=sharing

Link to GeoCLIM-formatted CHIRPS data at 0.2 degrees: https://docs.google.com/file/d/0B_H-a39WNYAUaGFuSXp0QlNUUmM/edit?usp=sharing

Rename the file to .zip and unzip it if you want to see what the _dataset file looks like and use it to make your own GeoCLIM archives from there.


From Chris - 20-09-2013:

Hi Master T - this is awesome. I am also including Pete, since he'll be handling your data requests (he knows this is really important stuff!). Pete - can you read T's message and help us assemble dekadal and monthly archives of the CHIRPs for africa?

Here are some suggested thoughts/responses:

a) BASIICS - I love it!

b) CHIRPS ftp - ftp://chg-ftpout.geog.ucsb.edu/pub/org/chg/products/CHIRPS/bils/

In the monthly directory you'll find and africa sub-directory ...


c) Regions and countries ... I'd suggest we start with SADC, CILLS, GHA, and the five countries in the EAC: Kenya, Uganda, Burundi, Rwanda and Tanzania. Diego can help you find admin units as necessary.


d) XP ... I think for now we should support xp. The performance hit for switching to windows 7 is pretty major, so I think it'll stick around for a long time.


One final thought is that we should try to bullet proof GHA-monthly rainfall time series and analysis - since that will be the primary focus of our training in October. The 2nd objeective would be the same, but with temperature data - Diego, can you help T find our Tmax and Tmin climatologies?


From Diego: 20-09-2013

I am exited to use the GeoCLIM, it sounds very good. I think the IRE name is simpler than the BAIISICS. but not - CHIRPS is not operational yet, it is updated up to July 2013 and there is not a clear time step for update. here is the link where the new data will be. ftp://chg-dub.geog.ucsb.edu/pub/org/chg/products/CHIRPS/bils/ - in term of countries, if the program makes the data available by default, the list is going to be long but I think it would be great to provide the user with a region. - yes, window XP is used in many places - OK, we'll be waiting patiently. Diego

Retrieved from "https://wiki.chc.ucsb.edu/index.php?title=Talk:Main_Page&oldid=1145"