Skip to content

Commit

Permalink
fix: reassemble the original XML file from the JSONs
Browse files Browse the repository at this point in the history
  • Loading branch information
mcarvin8 committed Apr 17, 2024
1 parent 3c31cd1 commit 0552b04
Show file tree
Hide file tree
Showing 9 changed files with 259 additions and 17 deletions.
39 changes: 36 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ A JavaScript package to disassemble then transform XML files into smaller JSON f

Large XML files can be a pain to mantain in version control. These files can contain thousands of lines and it becomes very difficult to track changes made to these files in a standard version control server like GitHub.

This package offers a way to break down large XML files into smaller JSON files which can be used to review changes in a format easier to digest.
This package offers a way to break down large XML files into smaller JSON files which can be used to review changes in a format easier to digest. When needed, the inverse class will reassemble the original XML file from the smaller JSON files.

This will parse and retain the following XML elements:

Expand All @@ -24,6 +24,8 @@ npm install xml2json-disassembler

## Usage

### XML 2 JSON

```typescript
/*
FLAGS
Expand All @@ -47,6 +49,8 @@ await handler.transform({
});
```

Disassemble then transform 1 or multiple XML files into JSON files. If the `xmlPath` is a directory, only the XMLs in the immediate directory will be processed. Each XML wiill be transformed into JSON files in new sub-directories using the XML's base name (everything before the first period in the file-name).

Example:

An XML file (`HR_Admin.permissionset-meta.xml`) with the following nested and leaf elements
Expand Down Expand Up @@ -111,6 +115,28 @@ will be disassembled into a sub-directory named `HR_Admin` as such:

<br>

### JSON 2 XML

```typescript
/*
FLAGS
- jsonPath: Path to the directory containing the JSON files to reassemble into 1 XML file (must be a directory).
- fileExtension: (Optional) Desired file extension for the final XML (default: `.xml`).
- postPurge: (Optional) Boolean value. If set to true, purge the disassembled directory containing JSON files after the XML is reassembled.
Defaults to false.
*/
import { JsonToXmlReassembler } from "xml2json-disassembler";

const handler = new JsonToXmlReassembler();
await handler.reassemble({
jsonPath: "test/baselines/HR_Admin",
fileExtension: "permissionset-meta.xml",
postPurge: true,
});
```

Reassemble all of the JSON files in a directory into 1 XML file. **Note:** You should only be reassembling JSON files created by the `XmlToJsonDisassembler` class for intended results. The reassembled XML file will be created in the parent directory of `jsonPath` and will overwrite the original file used to create the original disassembled directories, if it still exists and the `fileExtension` flag matches the original file extension.

## Logging

By default, the package will not print any debugging statements to the console. Any error or debugging statements will be added to a log file, `disassemble.log`, created in the same directory you are running this package in. This file will be created when running the package in all cases, even if there are no errors. I recommend adding `disassemble.log` to your `.gitignore` file.
Expand Down Expand Up @@ -143,14 +169,21 @@ import { XmlToJsonDisassembler, setLogLevel } from "xml2json-disassembler";

setLogLevel("debug");

const handler = new XmlToJsonDisassembler();
await handler.transform({
const disassembleHandler = new XmlToJsonDisassembler();
await disassembleHandler.transform({
xmlPath: "test/baselines/general",
uniqueIdElements:
"application,apexClass,name,externalDataSource,flow,object,apexPage,recordType,tab,field",
prePurge: true,
postPurge: true,
});

const reassembleHandler = new JsonToXmlReassembler();
await reassembleHandler.reassemble({
jsonPath: "test/baselines/HR_Admin",
fileExtension: "permissionset-meta.xml",
postPurge: true,
});
```

## Template
Expand Down
10 changes: 10 additions & 0 deletions src/helpers/types.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
"use strict";

export const INDENT = " ";

export const XML_PARSER_OPTION = {
commentPropName: "!---",
ignoreAttributes: false,
Expand All @@ -12,6 +14,14 @@ export const XML_PARSER_OPTION = {
cdataPropName: "![CDATA[",
};

export const JSON_PARSER_OPTION = {
...XML_PARSER_OPTION,
format: true,
indentBy: INDENT,
suppressBooleanAttributes: false,
suppressEmptyNode: false,
};

export interface XmlElement {
[key: string]: string | XmlElement | string[] | XmlElement[];
}
1 change: 1 addition & 0 deletions src/index.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import { getLogger, configure } from "log4js";
export { XmlToJsonDisassembler } from "./service/xml2jsonDisassembler";
export { JsonToXmlReassembler } from "./service/json2xmlReassembler";

export const logger = getLogger();

Expand Down
19 changes: 19 additions & 0 deletions src/service/deleteReassembledXML.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
"use strict";

import { stat, readdir, rm } from "node:fs/promises";
import { join } from "node:path";

export async function deleteReassembledXML(
disassembledPath: string,
): Promise<void> {
const files = await readdir(disassembledPath);
for (const file of files) {
const filePath = join(disassembledPath, file);
const fileStat = await stat(filePath);
if (fileStat.isFile() && filePath.endsWith(".xml")) {
await rm(filePath);
} else if (fileStat.isDirectory()) {
await deleteReassembledXML(filePath);
}
}
}
8 changes: 4 additions & 4 deletions src/service/disassembleHandler.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,14 @@ import { DisassembleXMLFileHandler } from "xml-disassembler";
export async function disassembleHandler(
xmlPath: string,
uniqueIdElements: string,
prepurge: boolean,
postpurge: boolean,
prePurge: boolean,
postPurge: boolean,
): Promise<void> {
const handler = new DisassembleXMLFileHandler();
await handler.disassemble({
xmlPath,
uniqueIdElements,
prePurge: prepurge,
postPurge: postpurge,
prePurge,
postPurge,
});
}
48 changes: 48 additions & 0 deletions src/service/json2xmlReassembler.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
"use strict";

import { stat, readdir } from "node:fs/promises";
import { join } from "node:path";

import { logger } from "@src/index";
import { reassembleHandler } from "@src/service/reassembleHandler";
import { transform2XML } from "@src/service/transform2XML";
import { deleteReassembledXML } from "@src/service/deleteReassembledXML";

export class JsonToXmlReassembler {
async reassemble(xmlAttributes: {
jsonPath: string;
fileExtension?: string;
postPurge?: boolean;
}): Promise<void> {
const {
jsonPath,
fileExtension = "xml",
postPurge = false,
} = xmlAttributes;
const fileStat = await stat(jsonPath);

if (fileStat.isFile()) {
logger.error(`The path ${jsonPath} is not a directory.`);
return;
} else if (fileStat.isDirectory()) {
await this.processFile(jsonPath);
}

await reassembleHandler(jsonPath, fileExtension, postPurge);
// delete XML files created during reassembly - this is needed if postPurge is false
if (!postPurge) await deleteReassembledXML(jsonPath);
}

async processFile(jsonPath: string): Promise<void> {
const files = await readdir(jsonPath);
for (const file of files) {
const filePath = join(jsonPath, file);
const fileStat = await stat(filePath);
if (fileStat.isFile() && filePath.endsWith(".json")) {
await transform2XML(filePath);
} else if (fileStat.isDirectory()) {
await this.processFile(filePath);
}
}
}
}
16 changes: 16 additions & 0 deletions src/service/reassembleHandler.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
"use strict";

import { ReassembleXMLFileHandler } from "xml-disassembler";

export async function reassembleHandler(
xmlPath: string,
fileExtension: string,
postpurge: boolean,
): Promise<void> {
const handler = new ReassembleXMLFileHandler();
await handler.reassemble({
xmlPath,
fileExtension,
postPurge: postpurge,
});
}
30 changes: 30 additions & 0 deletions src/service/transform2XML.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
"use strict";

import { readFile, writeFile } from "node:fs/promises";
import { XMLBuilder } from "fast-xml-parser";

import { logger } from "@src/index";
import { JSON_PARSER_OPTION, INDENT } from "@src/helpers/types";

export async function transform2XML(
jsonPath: string,
indentLevel: number = 0,
): Promise<void> {
const jsonString = await readFile(jsonPath, "utf-8");
const jsonObject = JSON.parse(jsonString);

// Remove XML declaration from JSON string
const xmlBuilder = new XMLBuilder(JSON_PARSER_OPTION);
const xmlString = xmlBuilder.build(jsonObject) as string;

// Manually format the XML string with the desired indentation
const formattedXml: string = xmlString
.split("\n")
.map((line: string) => `${" ".repeat(indentLevel * INDENT.length)}${line}`)
.join("\n")
.trimEnd();

const xmlPath = jsonPath.replace(/\.json$/, ".xml");
await writeFile(xmlPath, formattedXml);
logger.debug(`${jsonPath} has been transformed into ${xmlPath}`);
}
105 changes: 95 additions & 10 deletions test/main.spec.ts
Original file line number Diff line number Diff line change
@@ -1,18 +1,26 @@
import { writeFile, rm } from "node:fs/promises";
import { resolve } from "node:path";
import { readdir, readFile, writeFile, rm } from "node:fs/promises";
import { strictEqual } from "node:assert";
import { resolve, join } from "node:path";
import { copy } from "fs-extra";

import { XmlToJsonDisassembler, setLogLevel, logger } from "../src/index";
import {
XmlToJsonDisassembler,
JsonToXmlReassembler,
setLogLevel,
logger,
} from "../src/index";

setLogLevel("debug");
const baselineDir: string = "test/baselines";
const mockDir: string = "mock";
let xml2jsonDisassemblerHandler: XmlToJsonDisassembler;
let json2xmlReassemblerHandler: JsonToXmlReassembler;

describe("main function", () => {
beforeAll(async () => {
await copy(baselineDir, mockDir, { overwrite: true });
xml2jsonDisassemblerHandler = new XmlToJsonDisassembler();
json2xmlReassemblerHandler = new JsonToXmlReassembler();
});

beforeEach(async () => {
Expand All @@ -27,35 +35,112 @@ describe("main function", () => {
await rm(mockDir, { recursive: true });
});

it("should disassemble & transform a general XML file (folder path) into JSON files.", async () => {
it("should disassemble & transform 1 XML file into JSON files.", async () => {
await xml2jsonDisassemblerHandler.transform({
xmlPath: "mock",
xmlPath: "mock/HR_Admin.permissionset-meta.xml",
uniqueIdElements:
"application,apexClass,name,externalDataSource,flow,object,apexPage,recordType,tab,field",
});

expect(logger.error).not.toHaveBeenCalled();
});
it("should disassemble & transform a general XML file (file path) into JSON files.", async () => {
it("should disassemble & transform a directory of XML files into JSON files.", async () => {
await xml2jsonDisassemblerHandler.transform({
xmlPath: "mock/HR_Admin.permissionset-meta.xml",
prePurge: true,
postPurge: true,
xmlPath: "mock",
uniqueIdElements:
"application,apexClass,name,externalDataSource,flow,object,apexPage,recordType,tab,field",
prePurge: true,
postPurge: true,
});

expect(logger.error).not.toHaveBeenCalled();
});
it("should reassemble the XML file.", async () => {
await json2xmlReassemblerHandler.reassemble({
jsonPath: "mock/HR_Admin",
fileExtension: "permissionset-meta.xml",
});

expect(logger.error).not.toHaveBeenCalled();
});
it("should test error condition (XML file path not provided).", async () => {
it("should reassemble the XML file with comments.", async () => {
await json2xmlReassemblerHandler.reassemble({
jsonPath: "mock/Numbers-fr",
fileExtension: "globalValueSetTranslation-meta.xml",
});

expect(logger.error).not.toHaveBeenCalled();
});
it("should reassemble the CDATA XML file.", async () => {
await json2xmlReassemblerHandler.reassemble({
jsonPath: "mock/VidLand_US",
fileExtension: "marketingappextension-meta.xml",
});

expect(logger.error).not.toHaveBeenCalled();
});
it("should reassemble the XML file with an array of leafs.", async () => {
await json2xmlReassemblerHandler.reassemble({
jsonPath: "mock/Dreamhouse",
fileExtension: "app-meta.xml",
});

expect(logger.error).not.toHaveBeenCalled();
});
it("should reassemble the XML file with attributes.", async () => {
await json2xmlReassemblerHandler.reassemble({
jsonPath: "mock/attributes",
});

expect(logger.error).not.toHaveBeenCalled();
});
it("should test disassemble error condition (XML file path not provided).", async () => {
let fakeFile = "mock/not-an-xml.txt";
fakeFile = resolve(fakeFile);
const fakeFileContents = "Testing error condition.";
await writeFile(fakeFile, fakeFileContents);
await xml2jsonDisassemblerHandler.transform({
xmlPath: fakeFile,
});
expect(logger.error).toHaveBeenCalled();
});
it("should test reassemble error condition (file path provided).", async () => {
const fakeFile = "mock/not-an-xml.txt";
await json2xmlReassemblerHandler.reassemble({
jsonPath: fakeFile,
});
await rm(fakeFile);
expect(logger.error).toHaveBeenCalled();
});
// This should always be the final test
it("should compare the files created in the mock directory against the baselines to confirm no changes.", async () => {
await compareDirectories(baselineDir, mockDir);
});
});

async function compareDirectories(
referenceDir: string,
mockDir: string,
): Promise<void> {
const entriesinRef = await readdir(referenceDir, { withFileTypes: true });

// Only compare files that are in the reference directory
for (const entry of entriesinRef) {
const refEntryPath = join(referenceDir, entry.name);
const mockPath = join(mockDir, entry.name);

if (entry.isDirectory()) {
// If it's a directory, recursively compare its contents
await compareDirectories(refEntryPath, mockPath);
} else {
// If it's a file, compare its content
const refContent = await readFile(refEntryPath, "utf-8");
const mockContent = await readFile(mockPath, "utf-8");
strictEqual(
refContent,
mockContent,
`File content is different for ${entry.name}`,
);
}
}
}

0 comments on commit 0552b04

Please sign in to comment.