More Oozie Learnings

I just finished moving the code from my last post from our dev and qa boxes to production. During the time I was running in qa I did not, but should have, done testing running the process as a cron style job. Running the process as a cron style job has taught me a lot about how Oozie handles time. I learned three key lessons:

  1. All the times entered in Oozie regardless of the time zone specified in the configuration file are in GMT
  2. ${coord:current(0)} was putting the prior day in because of when it was running with relationship to GMT. Once I figured out the GMT thing I had to change to ${coord:current(-1)}.
  3. Manipulating the initial-instance variable for the data sets allowed me to back fill prior days. In oder to give our DWH team more test data I set it a week prior to the first actual run I wanted to do.

Tonight I will have another test run, fingers crossed everything should work well.

Advertisements
This entry was posted in Data Warehouse, Hadoop. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s