Welcome, Guest
Username: Password: Remember me
HELPDESK

Here we can describe more what should be posted here

TOPIC: gl problem with CY43 grib2 files

gl problem with CY43 grib2 files 1 year 2 months ago #2475

I have encountered a problem when trying to pick data for mast verification from CY43 grib2 files with gl_grib_api. With present operative MEPS grib2 files the system works, but when trying to read a CY43 grib2 history file I get an error

Starting gl_grib_api
gl runs with OpenMP threads: 24
Read namelist number 1
Using namelist :nameinterp_mast
Table of content
No such field found in file 2 unknown : 00000 heightAboveGround 20201009@0600+000h00m-001h00m Direct solar exposure (J m-2)

I get this error even if I try to pick only temperature.

If I run gl_grib_api without the namelist to get parameter info, I find that instead of the “Direct solar irradiance (W m-2)” I get from the present operative files and which I would need, from the CY43 grib2 file I get a parameter “Direct solar exposure (J m-2)”.

I checked the ECCodes files I am using and found that in the file

/opt/fmi/eccodes/2.8.2-1/share/eccodes/definitions/grib2/localConcepts/eswi/shortName.def

there indeed are two definitions

/opt/fmi/eccodes/2.8.2-1/share/eccodes/definitions/grib2/tables/16/4.2.3.6.table:2 2 Direct solar irradiance (W m-2)

/opt/fmi/eccodes/2.8.2-1/share/eccodes/definitions/grib2/tables/16/4.2.3.6.table:3 3 Direct solar exposure (J m-2)

Why is it that I get a different parameter from the present and CY4.3 files?

The gl command I use is

gl_grib_api -n nameinterp_mast -igd $FILE -s

the namelist file is attached

Markku

File Attachment:

File Name: nameinterp_mast.txt
File Size: 2 KB

gl problem with CY43 grib2 files 1 year 2 months ago #2476

  • Ulf Andrae
  • Ulf Andrae's Avatar
  • OFFLINE
  • Administrator
  • Posts: 308
  • Thank you received: 35
Markku,
Before digging further please ensure two things.

1. Use gl from a recent CY43 version, not gl_grib_api.
2. Always make sure you have our local definitions in the ECCODES_DEFINITON_PATH. E.g. at for a recent CY43 experiment on cca we have

ECCODES_DEFINITION_PATH=/scratch/ms/se/snh/hm_home/43_ref_6517/lib/const/eccodes/definitions:/scratch/ms/se/snh/hm_home/43_ref_6517/lib/util/gl/definitions:/usr/local/apps/eccodes/2.12.5/GNU/73/share/eccodes/definitions

The first part of the path is to accomodate some local bufr tables, the second our GRIB definitions and the last the default ECCODES native tables.

It the above mentioned doesn't help we'll have to dig further.

Ulf

gl problem with CY43 grib2 files 1 year 2 months ago #2477

Ulf,

Great, thanks for your quick answer! I am now using the same gl (grib_api) and ECCodes we are using in the present operative environment, so that may indeed be the cause.

Markku

gl problem with CY43 grib2 files 1 year 1 month ago #2478

I have now changed the setup by introducing ECCODES definitions and gl from the present MNWC_prepop_cy43 setup in voima:

export ECCODES_DEFINITION_PATH=/data/users/metcoop/hm_home/MNWC_preop_cy43_v1/util/gl/definitions/:/opt/fmi/eccodes/2.3.0-1/share/eccodes/definitions

The problem seems to be with parameters 'swavr' and 'lwavr'. If I remove them, there is no errors and the output seems to be OK, but if I include them, I get errors

gl runs with OpenMP threads: 24
Read namelist number 1

Using namelist :nameinterp_mast

Read namelist number 2
Gridtype differ
lambert regular_ll
Dimension differ
949 1
1069 1
Geografy differ
Projection parameters differ
63.299999999999997 0.0000000000000000
63.299999999999997 0.0000000000000000
15.000000000000000 0.0000000000000000
Table of content
Not found in file 1 *** sshf :*** 00000 heightAboveGround -1@****+***h00m # (#)
Ambigous fields, please check readkey%tri or readkey%pdtn
-> 1 *** swavr :*** 00000 heightAboveGround -1@****+***h00m # (#)
-> 2 swavr : 00000 heightAboveGround 20200930@0600+000h00m-001h00m (00/15) Downward short-wave radiation flux (W m-2)
-> 2 swavr : 00000 heightAboveGround 20200930@0600+001h00m (00/15) Downward short-wave radiation flux (W m-2)


or

gl runs with OpenMP threads: 24
Read namelist number 1

Using namelist :nameinterp_mast

Read namelist number 2
Gridtype differ
lambert regular_ll
Dimension differ
949 1
1069 1
Geografy differ
Projection parameters differ
63.299999999999997 0.0000000000000000
63.299999999999997 0.0000000000000000
15.000000000000000 0.0000000000000000
Table of content
Not found in file 1 *** sshf :*** 00000 heightAboveGround -1@****+***h00m # (#)
Not found in file 1 *** dni :*** 00000 heightAboveGround -1@****+***h00m # (#)
Ambigous fields, please check readkey%tri or readkey%pdtn
-> 1 *** lwavr :*** 00000 heightAboveGround -1@****+***h00m # (#)
-> 2 lwavr : 00000 heightAboveGround 20200930@0600+000h00m-001h00m (00/15) Upward long-wave radiation flux (W m-2)
-> 2 lwavr : 00000 heightAboveGround 20200930@0600+001h00m (00/15) Upward long-wave radiation flux (W m-2)

If run gl just to get the parameters, there indeed seems to be two entries for those parameters. How do I choose between them? The error message suggests to check the 'readkey%tri' variable. I am using 011 for it, as was with the earlier gl(_grib_api) pick. I don't see info about that in the parameter output produced by gl.

I also tried by using eccodes/2.8.2-1 (which I wad using earlier) but in vain, got the same error.

The namelist I am using was attached in my first mail.

Markku
Last Edit: 1 year 1 month ago by Markku Kangas.

gl problem with CY43 grib2 files 1 year 1 month ago #2479

I just noticed that with this new gl, I have the same problem with the original (operative) gribs, which work with the old gl. Perhaps something should be changed in the namelist?

Markku

gl problem with CY43 grib2 files 1 year 1 month ago #2480

Hi Markku,

is this still a problem?
Looking at the grib2 files I see for example 2 swavr fields with typeOfFirstFixedSurface=103, which would be the "heighAboveGround". One is the instant the other the accum.
I think the problem is in util/gl/definitions/grib2/localConcepts/mcoop/shortName.def where forecastTime=0; is missing in definition of 'swavr'.
util/gl/definitions/grib2/localConcepts/251/shortName.def has it.

David
The following user(s) said Thank You: Carl Fortelius

gl problem with CY43 grib2 files 1 year 1 month ago #2481

Hi David,

Thanks for your answer. And yes, it is still a problem. I have also noticed that there is a problem with the parameter shortnames. If I run the (new) gl just to get parameters in the file, I see that some of them are missing, but some are mentioned up to three times in the gl output, so there seems also be some ambiguity. Because the the new gl with new definitions does not work with the cy40 gribs, it seems probable that the problem really is in the definitions.

I'll try to replace the def file with the one you suggested.

gl problem with CY43 grib2 files 1 year 1 month ago #2482

David,

What is the "base path" for the util directory, is it

/data/users/metcoop/hm_home/MNWC_preop_cy43_v1/[util...]

that I am using now? If so, it did not help; I tried by replacing the def link as you suggested, but now got an error:

gl runs with OpenMP threads: 24
Read namelist number 1

Using namelist :nameinterp_mast

Second identical field found

Problem here is that the scripts only define the path up to .../gl/definitions/; I must check the scripts whether the rest of the path is hard-coded somewhere in the script (have not originally made them).

Furthermore, if I look at the directory

ls ...util/gl/definitions/grib2/localConcepts/

lrwxrwxrwx 1 hirlam2 hirlam 5 Mar 28 2019 251 -> mcoop/
drwxr-xr-x 2 hirlam2 hirlam 4096 Jun 12 2019 86/
drwxr-xr-x 2 hirlam2 hirlam 4096 May 2 2019 eidb/
lrwxrwxrwx 1 hirlam2 hirlam 3 Jun 12 2019 fmi -> 86//
drwxr-xr-x 2 hirlam2 hirlam 4096 May 2 2019 mcoop/

I see that 251 is linked to directory mcoop, so file shortName.def is identical both in 251 and mcoop. Am I in wrong directory (see the "base path" in the beginning)?

It strongly seems that I must go back to the drawing board with the cy43 data... I am in a bit of hurry with building and reporting this system, so I think I'll postpone the handling cy43 data for a while and prepare the system without it.

gl problem with CY43 grib2 files 1 year 1 month ago #2483

I now managed to make the extraction to work. On David's suggestion I added

forecastTime=0

for parameters swavr and lwavr in the gl definition file

shortName.def

That did the trick. David, thanks once more!

gl problem with CY43 grib2 files 1 year 1 month ago #2484

  • Ulf Andrae
  • Ulf Andrae's Avatar
  • OFFLINE
  • Administrator
  • Posts: 308
  • Thank you received: 35
Marrku, David,

Sorry for not being very helpful. I don't understand the problem with the grib2 tables since 251 has always pointed to mcoop? The file has three swavr

[a000864@c21883 gl_testbed]$ ~/local/eccodes-2.17.0/bin/grib_ls fc2019100100+003grib2 -w shortName=swavr -p date,stepRange,stepType,typeOfLevel

date stepRange stepType typeOfLevel
20191001 0-3 accum heightAboveGround
20191001 0-3 accum nominalTop
20191001 3 instant heightAboveGround
3 of 840 messages in fc2019100100+003grib2

And I assume you're interested in the accumulated one?

In any case there are some problems with the namelist

- readkey%tri should be replace by readkey%pdtn, but the numbers are correct. This is poorly documented and I would have used stepType if it wasn't undefined for some GRIB1 cases. I'll see if I can make a more user friendly construction without breaking either GRIB1/GRIB2 handling.

- sshf -> shf, dni -> dne

- Don't use heightAboveGroundHighPrecision since it's not defined in GRIB2. It happen to work in gl but nowadays the standard surface reference works as well.
PPPKEY(1:)%levtype = 'heightAboveGround','heightAboveGround',
PPPKEY(1:)%level = 32,32,


Ulf
The following user(s) said Thank You: Markku Kangas

gl problem with CY43 grib2 files 1 year 1 month ago #2485

Ulf,

Great, thanks for your answer and clarifications! I'll proceed with them.

Markku
Time to create page: 0.103 seconds