#103. Also documentation.

README.md

@@ -5,11 +5,7 @@
 
 # Carnival
 
-*It's a party of information!*
-
-**Carnival** Carnival is an open source JVM data unification framework that allows for a large variety of extract, transform, and load (ETL), integration, and analysis tasks related to relational data and property graphs. Some key functionality includes aggregation of data from disparate sources into a unified property graph and tools to reason over and interact with graph data using bounded operations. Carnival includes an architecture for tracking the provenance of data and providing evidence chains for conclusions or reasoning made on those data.
-
-While Carnival is a general purpose tool that is domain agnostic, there are extensions ([carnival-clinical](https://github.com/carnival-data/carnival-clinical), [carnival-openspecimen](https://github.com/carnival-data/carnival-openspecimen)) that provide models and algorithms specific to the clinical biobanking domain.
+**Carnival** Carnival is an open source JVM data unification framework that allows for a large variety of extract, transform, and load (ETL), integration, and analysis tasks related to relational data and property graphs. Some key functionality includes a graph model specification, the aggregation of data from disparate sources into a unified property graph, and tools to reason over and interact with graph data using bounded operations. 
 
 ## External Resources
 
@@ -24,7 +20,7 @@ While Carnival is a general purpose tool that is domain agnostic, there are exte
 
 ## <a name="overview"></a> Overview
 
-Carnival has three principal components: a graph modeling architecture, a caching facility for aggregating data from disparate data sources, and a framework for implementing graph algorithms.  The graph modeling architecture is a layer over Java enumerations and Tinkerpop that allow a graph model to be modeled and consumed by Tinkerpop traversals.  The caching facility supports the aggregation and caching of data from relational database and JSON API sources.  The graph algorithm framework provides a structured way to define and execute algorithms that operate over the property graph.
+Carnival has three principal components: a graph modeling architecture, a caching facility for aggregating data from disparate data sources, and a framework for implementing graph algorithms.  The graph modeling architecture is a layer over Java enumerations and Tinkerpop that allow a graph to be modeled and consumed by Tinkerpop traversal steps.  The caching facility supports the aggregation and caching of data from relational database and JSON API sources.  The graph algorithm framework provides a structured way to define and execute algorithms that operate over the property graph.
 
 
 ## <a name="packages"></a> Packages
@@ -34,6 +30,7 @@ Carnival has three principal components: a graph modeling architecture, a cachin
 Name | Description
 --- | ---
 carnival-core | Basic Carnival framework. Implements the basic Carnival framework classes (vines, reapers, reasonsers, etc). Defines the basic carnival graph schema (processes, databases). [Core model](https://github.com/pmbb-ibi/carnival/blob/master/app/carnival-core/src/main/groovy/carnival/core/graph/Core.groovy)
+carnival-vine | Mechanisms to faciliate the aggregation of data with data caching.  
 [carnival-graph](app/carnival-graph/README.md) | Framework for defining carnival graph schemas (vertex and edge definitions). Contains the basic vertex, edge, and property classes.
 [carnival-util](app/carnival-util/README.md) | Standalone package that contains utility and helper classes such as data tables, reports, and SQL utilties, which are primarily used for dealing with relational data.
 [carnival-gradle](app/carnival-gradle/README.md) | Gradle plugin for building Carnival applications and libraries.
@@ -45,7 +42,6 @@ carnival-core | Basic Carnival framework. Implements the basic Carnival framewor
 
 Name | Description
 --- | ---
-[carnival-clinical](https://github.com/carnival-data/carnival-clinical) | Extension of carnival-core for clinical data and biobanking operations. Contains model extensions for concepts such as patients, patient cohorts and healthcare encounters. Implements algorithms for generating case-control patient cohorts. [Clinical Model Extension](https://github.com/carnival-data/carnival-clinical/blob/main/src/main/groovy/carnival/clinical/graph/Clinical.groovy)
 [carnival-openspecimen](https://github.com/carnival-data/carnival-openspecimen) | Extension of carnival-core that implements vines for interfacing with [OpenSpecimen](https://www.openspecimen.org/) inventory management system.
 
 ## <a name="contribution-guide"></a> Contribution Guide

app/build.gradle

@@ -148,6 +148,7 @@ subprojects {
     apply plugin: "maven-publish"
     apply plugin: "signing"
     publishToMavenLocal.dependsOn 'printVersions'
+    tasks.withType(Sign) { onlyIf { isReleaseVersion } }
     publishing {
         repositories {
             maven {

app/carnival-core/build.gradle

@@ -66,19 +66,11 @@ publishing {
     }
 }
 
-
 signing {
-    required { 
-        isReleaseVersion && 
-        gradle.taskGraph.hasTask("publish") &&
-        !gradle.taskGraph.hasTask("publishToMavenLocal")
-    }
-
     sign publishing.publications.mavenGroovy
 }
 
 
-
 ///////////////////////////////////////////////////////////////////////////////
 // dependencies
 ///////////////////////////////////////////////////////////////////////////////

app/carnival-core/src/test/groovy/carnival/core/CarnivalNeo4jSpec.groovy

@@ -129,7 +129,7 @@ class CarnivalNeo4jSpec extends Specification {
             e.printStackTrace()
             def pluginDir = CarnivalNeo4jConfiguration.defaultConfiguration().gremlin.neo4j.conf.dbms.directories.plugins
             println "in order to run APOC, the APOC library must be present on the file system and configured in the application configuration."
-            println "has carnival.home been set?  or a configuration otherwise provided?"
+            println "has a configuration been provided?"
             println "is the following plugin directory valid? ${pluginDir}"
         }
 

app/carnival-gradle/build.gradle

@@ -1,9 +1,18 @@
 /**
  * carnival-gradle
  *
- */
+*/
 
 
+///////////////////////////////////////////////////////////////////////////////
+// plugins needed to publish to the Gradle plugin repository
+///////////////////////////////////////////////////////////////////////////////
+
+plugins {
+    id 'java-gradle-plugin'                           
+    id 'com.gradle.plugin-publish' version '1.1.0'   
+}
+
 ///////////////////////////////////////////////////////////////////////////////
 // properties file
 ///////////////////////////////////////////////////////////////////////////////
@@ -75,17 +84,35 @@ publishing {
 }
 
 signing {
-    required { 
+    /*required { 
         isReleaseVersion && 
         gradle.taskGraph.hasTask("publish") &&
         !gradle.taskGraph.hasTask("publishToMavenLocal")
-    }
-
+    }*/
     sign publishing.publications.mavenGroovy
 }
 
 
+///////////////////////////////////////////////////////////////////////////////
+// Gradle publishing
+///////////////////////////////////////////////////////////////////////////////
+
+pluginBundle {
+    website = 'https://carnival-data.github.io/carnival/' 
+    vcsUrl = 'https://github.com/carnival-data/carnival/' 
+    tags = ['carnival', 'graph', 'data']     
+}
 
+gradlePlugin {
+    plugins { 
+        greetingsPlugin { 
+            id = 'io.github.carnival-data.carnival' 
+            displayName = 'Carnival' 
+            description = 'The Carnival plugin adds the necessary dependencies for an application that relies on Carnival' 
+            implementationClass = 'carnival.gradle.CarnivalLibraryPlugin'
+        }
+    }
+}
 
 ///////////////////////////////////////////////////////////////////////////////
 // dependencies

app/carnival-gradle/src/main/groovy/carnival/gradle/CarnivalApplicationPlugin.groovy

@@ -1,12 +1,17 @@
 package carnival.gradle
 
 
-
 import org.gradle.api.Plugin
 import org.gradle.api.Project
 
 
-
+/**
+ * A Gradle plugin that was intended to make it easier to create applications
+ * that rely on Carnival.  It is no longer used.
+ *
+ * @Deprecated
+ *
+ */
 class CarnivalApplicationPlugin implements Plugin<Project> {
 
     void apply(Project project) {

app/carnival-gradle/src/main/groovy/carnival/gradle/CarnivalLibraryPlugin.groovy

@@ -1,12 +1,15 @@
 package carnival.gradle
 
 
-
 import org.gradle.api.Plugin
 import org.gradle.api.Project
 
 
-
+/**
+ * A simple Gradle plugin to add a consistent set of dependencies to an 
+ * application that relies on Carnival.
+ *
+ */
 class CarnivalLibraryPlugin implements Plugin<Project> {
 
     void apply(Project project) {
@@ -19,12 +22,12 @@ class CarnivalLibraryPlugin implements Plugin<Project> {
         def neo4JavaDriverVersion = versions.getString("neo4JavaDriverVersion")
         def carnivalVersion = versions.getString("carnivalVersion")
 
-        println "[CarnivalLibrary] Java version: ${System.getProperty('java.version')}"
-        println "[CarnivalLibrary] Groovy version: ${groovyVersion}"
-        println "[CarnivalLibrary] Gremlin version: ${gremlinVersion}"
-        println "[CarnivalLibrary] Neo4j Tinkerpop version: ${neo4jTinkerpopVersion}"
-        println "[CarnivalLibrary] Neo4 Java Driver version: ${neo4JavaDriverVersion}"
-        println "[CarnivalLibrary] Carnival version: ${carnivalVersion}"
+        println "[Carnival] Java version: ${System.getProperty('java.version')}"
+        println "[Carnival] Groovy version: ${groovyVersion}"
+        println "[Carnival] Gremlin version: ${gremlinVersion}"
+        println "[Carnival] Neo4j Tinkerpop version: ${neo4jTinkerpopVersion}"
+        println "[Carnival] Neo4 Java Driver version: ${neo4JavaDriverVersion}"
+        println "[Carnival] Carnival version: ${carnivalVersion}"
 
         // apply dependencies
         project.dependencies {
@@ -45,6 +48,7 @@ class CarnivalLibraryPlugin implements Plugin<Project> {
             implementation("io.github.carnival-data:carnival-util:${carnivalVersion}")
             implementation("io.github.carnival-data:carnival-graph:${carnivalVersion}")
             implementation("io.github.carnival-data:carnival-core:${carnivalVersion}")
+            implementation("io.github.carnival-data:carnival-vine:${carnivalVersion}")
         }
     }
 }

app/carnival-graph/build.gradle

@@ -71,18 +71,11 @@ publishing {
 }
 
 signing {
-    required { 
-        isReleaseVersion && 
-        gradle.taskGraph.hasTask("publish") &&
-        !gradle.taskGraph.hasTask("publishToMavenLocal")
-    }
-
     sign publishing.publications.mavenGroovy
 }
 
 
 
-
 ///////////////////////////////////////////////////////////////////////////////
 // dependencies
 ///////////////////////////////////////////////////////////////////////////////

app/carnival-util/build.gradle

@@ -71,17 +71,10 @@ publishing {
 }
 
 signing {
-    required { 
-        isReleaseVersion && 
-        gradle.taskGraph.hasTask("publish") &&
-        !gradle.taskGraph.hasTask("publishToMavenLocal")
-    }
-
     sign publishing.publications.mavenGroovy
 }
 
 
-
 ///////////////////////////////////////////////////////////////////////////////
 // dependencies
 ///////////////////////////////////////////////////////////////////////////////

app/carnival-vine/build.gradle

@@ -84,12 +84,6 @@ publishing {
 }
 
 signing {
-    required { 
-        isReleaseVersion && 
-        gradle.taskGraph.hasTask("publish") &&
-        !gradle.taskGraph.hasTask("publishToMavenLocal")
-    }
-
     sign publishing.publications.mavenGroovy
 }
 

docs/index.md

@@ -152,7 +152,7 @@ class MyDomain {
      * and LAST.
      * 
      */ 
-    @VertexDefinition
+    @VertexModel
     static enum VX {
         PERSON,
 
@@ -170,7 +170,7 @@ class MyDomain {
      * have domain and range restrictions.
      * 
      */ 
-    @EdgeDefinition
+    @EdgeModel
     static enum EX {
         IS_NAMED(
             domain:[VX.PERSON],
@@ -183,7 +183,7 @@ class MyDomain {
      * The following enum defines allowed properties.
      * 
      */ 
-    @PropertyDefinition
+    @PropertyModel
     static enum PX {
         FIRST,
         LAST,

docs/production-builds.md

@@ -1,28 +1,64 @@
 # Production Builds
 
+## Publishing to Maven Local
+
+Carnival artifacts can be published to your local Maven repository usually found on your file system in the directory `~/.m2` via the following command. 
+
+```Shell
+./gradlew :carnival-gradle:publishToMavenLocal 
+```
+
 ## Publishing to Maven
 
-* Maven Repository Manager: https://s01.oss.sonatype.org/#welcome
-* Maven: https://search.maven.org/artifact/io.github.carnival-data/carnival-core
+Carnival artifacts are published to [Maven](https://search.maven.org/search?q=io.github.carnival-data) via the [Nexus Repository Manager](https://s01.oss.sonatype.org).
 
 ### Configuration
-Copy `.env-template` to `.env` and update the file with your maven central credentials and private key information. The signing file should be .gpg format, and SIGNING_KEY_ID is usually the last 8 digits of the fingerprint. More detail about the signing plugin available [here](https://docs.gradle.org/7.4.1/userguide/signing_plugin.html#sec:signatory_credentials).
+Copy `.env-template` to `.env` and update the file with your maven central credentials and private key information. The signing file should be **.gpg** (not **.asc**) format.  `SIGNING_KEY_ID` is usually the last 8 digits of the fingerprint. More detail about the signing plugin available [here](https://docs.gradle.org/7.4.1/userguide/signing_plugin.html#sec:signatory_credentials).
 
 
 ### Publish to Snapshot Repository
-When the carnivalVersion specified in `app\gradle.properties` ends with "-SNAPSHOT", the package will be published to the snapshot repository. Previous releases with the same version can be overwritten.
-```
+When the carnivalVersion specified in `app/gradle.properties` ends with "-SNAPSHOT", the package will be published to the snapshot repository. Previous releases with the same version can be overwritten.
+
+#### Docker
+
+```Shell
 docker-compose -f docker-compose-publish-maven.yml up
 ```
 
+#### Gradle
+
+From the `app/` directory:
+
+```Shell
+source ../.env
+./gradlew :carnival-gradle:publishAllPublicationsToCentralRepository 
+```
+
 ### Publish Release Versions
 If the version number does not end with "-SNAPSHOT", the package will be published to the staging repository and must be manually approved.
 
-1. Run the following to publish to the staging repository:
+#### Docker
+
+Run the following to publish to the staging repository:
 ```
 docker-compose -f docker-compose-publish-maven.yml up
 ```
 
+#### Gradle
+
+From the `app/` directory:
+
+
+```Shell
+source ../.env
+```
+
+```Shell
+./gradlew publishAllPublicationsToCentralRepository -Psigning.secretKeyRingFile=${SIGNING_PRIVATE_DIR}/${SIGNING_PRIVATE_FILE} -Psigning.password=${SIGNING_PRIVATE_KEY_PASSWORD} -Psigning.keyId=${SIGNING_KEY_ID} -Pcentral.user=${CENTRAL_USER} -Pcentral.password=${CENTRAL_PASSWORD} --no-daemon --console=plain
+```
+
+#### Publish The Release
+
 1. Log into the [Maven Nexus Repository Manager](https://s01.oss.sonatype.org/#welcome)
 
 1. Click "Staging Repositories" on the left. The repository that was just published should be visible.
@@ -32,7 +68,8 @@ docker-compose -f docker-compose-publish-maven.yml up
 1. If the validation is successfull, click "Release" to publish the release.
 
 
-### Publishing Production Builds to Github (Deprecated)
+
+## Publishing Production Builds to Github (Deprecated)
 
 Production images are published to Github packages. In order to publish an image, you will need to create a Github personal access token with appropriate permissions to manage github packages (see Github Packages documentation for details). Then create local environment variables **GITHUB_USER** and **GITHUB_TOKEN** with your github user and personal access token.
 Once authorization has been set up, the procedure to publish production builds is: