Skip to content

Releases: scm-spain/slippin-jimmy

Hotfix 3.0.1/Fixed Tlacuilo for single .yml file

02 Jan 14:10
a9292fd
Compare
Choose a tag to compare

Hotfix 3.0.1/Fixed Tlacuilo for single .yml file:

  • Now Tlacuilo works correctly when a single .yml file is provided to the -w parameter, instead of autodiscovering them when a folder path is provided.

Release 3.0.0/Added 'local' mode for Hersir

21 Dec 12:17
4ee5d41
Compare
Choose a tag to compare
  • Removed default_configuration.py to avoid coupling with our templates definition
  • Removed swamp & hdfs_deploy_folder params from Tlacuilo to avoid coupling, they should be used as --extra parameters now
  • Added -l --local mode to run Hersir from inside EMR (no AWS or ssh calls should be used)
  • Added -j --job-file-name on Cooper to be able to avoid interactive mode if one knows which .properties file should be called by Oozie
  • Now if "hdfs_deploy_folder" variablre is present on the configuration files, it will be used as the HDFS deploy folder for Anabasii. If not, you can still provide the deploy folder via -d HDFS_DEPLOY_FOLDER.
  • Updated requirements.txt to add new cx_Oracle version 6.1 (Dec 2017)

Feature/2.1.0: Tlacuilo support infering EMR config

20 Nov 15:30
e396847
Compare
Choose a tag to compare

Feature/2.1.0: Tlacuilo support infering EMR config

  • Now the Tlacuilo module asks for EMR ID if you specify the --interactive flag, so there's no longer need to pass EMR-related parameters (like private IP) when calling Hersir with --extra parameters.

Release 2.0

15 Nov 08:41
e0b8ea1
Compare
Choose a tag to compare

Release 2.0

  • Deprecated all Hive & Sqoop parameters from Scribe generation. If still want to be used, you should go back to tag 1.7.0.

feature 1.7.0/Adding .py spark scripts on workflows

11 Oct 14:50
Compare
Choose a tag to compare
  • Added new "auto" feature for Tlacuilo/Hersir: if there are any .py files inside the folders where the .yml selected files are located, they'll be "packed" on the generated workflow, under workflow_root_folder/lib/ path. This way one can start using pyspark scripts on new workflows easily.

Hotfix/1.6.6 Updated required pymssql version

10 Oct 07:17
Compare
Choose a tag to compare
  • Updated required pymssql version to 2.1.3 to avoid errors when installing slippinj.

hotfix/1.6.5 Updated required psycopg2 versions

09 Oct 14:27
Compare
Choose a tag to compare
  • Updated requirements.txt file to force newer psycopg2 version to avoid pg_config issues w/ Docker

hotfix/1.6.4 Force required libs versions

09 Oct 12:24
Compare
Choose a tag to compare
  • Modified requirements.txt to enforce lib versions to avoid non retro-compatible upgrades.
  • Removed apt-get update from travis.yml to fasten the build.

Hotfix 1.6.3/Postgresql driver adapted to be able to extract data from Amazon Redshift

14 Sep 15:52
Compare
Choose a tag to compare
Merge pull request #32 from scm-spain/hotfix/docker-oracle

Fix on Dockerfile:

Hotfix 1.6.2/Oracle driver fix: system columns being added on .yml

28 Jul 08:53
Compare
Choose a tag to compare
  • System columns being added on .yml files when they shouldn't.
  • Added check for numeric types to determine if should be mapped as double or bigint on Hive.