tinker: template config objects more customisation

[technophile-04]

NOTE: This is more like a discussion, brainstorming PR and will create a PR for other files which doesn't have variable in object in them. But we will encounter this problem in future as well

Goal:

The goal was to allow more customization for templates file (specially for this PR, templates which have config object like structure)

Was planning to have just two args for this config object like template

1. preConfigContent (type: string code snippet) => Allows developer to add imports/custom pre config object setup logic if they want
2. configObj (type: Object) => This object will be merged with default SE-2 config object.

Problem:

When having config objects whose key values are variable which comes from preConfigContent code snippet, we can't seem to access it during runtime and CLI errors out.

For eg: checkout this PR file changes hardhat.config.template.mjs. Since deployerPrivateKey is variable which comes from preConfigContent code snippet (its string) while resolving that object, node js runtime doesn't have any idea about deployerPrivateKey in the object and it errors out.

Example error: yarn build && yarn cli ../test-blah -s hardhat --skip :

Error image:

*Similarly, if people create a .arg.mjs which has a uses certain variable in configObj then CLI will error again too.

Example `hardhat.config.ts.args.mjs`

export const preConfigContent = `
// Custom variables
const CUSTOM_API_KEY = process.env.CUSTOM_API_KEY;
`;

export const configOverride = {
networks: {
  hardhat: {
    forking: {
      blockNumber: 1234567
    }
  },
  customNetwork: {
    url: "https://custom.network",
    accounts: [deployerPrivateKey],
    verify: {
      etherscan: {
        apiUrl: "https://api.custom-explorer.io",
        apiKey: etherscanApiKey,
      }
    }
  }
},
};

We are getting ReferenceErrror: deployerPrivateKey not defined because that variable isn't present in JS scope/context.

Hacky solution explored:

Didn't find any good solutions to this, found a very hacky solution but not sure if we should go with it or not and make it completely working.

Injecting this "not defined" variables in JS context virtually on run time.

Checkout this very hacky code snippet which tries to inject not defined variables in context for .args.mjs file (thanks to claude):

create-eth/src/tasks/copy-template-files.ts

Lines 44 to 98 in 2ee1b9e

	async function loadArgsFile(argsFileUrl: string): Promise<Record<string, any>> {
	const filePath = fileURLToPath(argsFileUrl);
	const content = await fs.promises.readFile(filePath, "utf8");
	createRequire(import.meta.url);
	// Explicitly typed import function
	const importFunction: ImportFunction = async (id: string) => {
	const module = await import(id);
	return module as Record<string, unknown>;
	};
	// Create base context object
	const baseContext: ModuleContext = {
	exports: {},
	import: importFunction,
	module: { exports: {} },
	__filename: filePath,
	__dirname: path.dirname(filePath),
	};
	// Create proxied context with type-safe getter
	const context = new Proxy(baseContext, {
	get: (target: ModuleContext, prop: string): unknown => {
	if (prop in target) {
	return target[prop];
	}
	return typeof prop === "string" ? prop : undefined;
	},
	});
	// Convert the content to an ES module style export
	const processedContent = content.replace(/export const/g, "module.exports.");
	// Wrap the content
	const wrappedContent = `
	(function(exports, import_, module, __filename, __dirname) {
	${processedContent}
	})`;
	// Create and run the script
	const script = new vm.Script(wrappedContent);
	const contextObj = vm.createContext(context);
	const moduleFunction = script.runInContext(contextObj) as ModuleFunction;
	moduleFunction.call(
	undefined,
	context.exports,
	context.import,
	context.module,
	context.__filename,
	context.__dirname,
	);
	return context.module.exports;

This still doesn't does the job completely since in final merged hardhat.config.ts you would that variables are still wrapped around quotes from hardhat-config extension but at least its able to parse .arg.mjs without CLI erroring out.

Need to handle .template.mjs still and its getting hard messy to inject context for this file.

Incase you want to test:

1. Clone the this repo in externalExtensions dir :

git clone https://github.com/technophile-04/hardhat-config

2. Switch to tinker-object-merge-v2 branch:

git switch tinker-object-merge-v2

3. yarn build and create a example instance:

yarn build && yarn cli ../test-blah -s hardhat -e hardhat-config --dev --skip

Other simple solution but not better DX:

We tell people to whenever they want to access the variable in object from preConfigContent snippet or SE-2 template instead of directly passing raw variable they pass string with certain special character like: 'account: ["$$${deployerPrivateKey}"]'. and in templating logic we can may strip that special character somehow.

NOTE: If an object has a comment it won't be present in the final merged config file.

---

I think initially while developing this templating engine we thought only strings would be passed but when objects are passed its like we have to come with hacky solutions and do some massaging as we had to do it here too https://github.com/scaffold-eth/create-eth/pull/145/files#r1850908537

Would love if anyone has better idea/suggestion 🙌

[rin-st]

I think we don't need hacky solution because.. it's too hacky and difficult to maintain

I played a bit with second solution with special characters and it works pretty good #175, see comments

So I think it's a way to go since it's much simpler

I also tried it with your extensions and it works

git clone https://github.com/technophile-04/hardhat-config

but change the file a bit to

Code

export const preConfigContent = `
// Custom variables
const CUSTOM_API_KEY = process.env.CUSTOM_API_KEY;
`;

export const configOverrides = {
  networks: {
    hardhat: {
      forking: {
        blockNumber: 1234567
      }
    },
    customNetwork: {
      url: "https://custom.network",
      accounts: ["$$$deployerPrivateKey"],
      verify: {
        etherscan: {
          apiUrl: "https://api.custom-explorer.io",
          apiKey: "$$$etherscanApiKey",
        }
      }
    }
  },
};