Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for provisioning macOS jenkins agents #38

Open
wants to merge 23 commits into
base: latest
Choose a base branch
from
Open
Changes from 22 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
b9c6f6a
Start macos recipe.
nuclearsandwich Mar 8, 2023
f8d3b99
Try to resolve error on deploy.
nuclearsandwich Mar 9, 2023
5086a09
Give up on managing users and try setting a default.
nuclearsandwich Mar 9, 2023
5a9f317
This default did not apply. See if this helps.
nuclearsandwich Mar 9, 2023
0c27dd2
Try running the command directly.
nuclearsandwich Mar 9, 2023
cfb3268
Swtich gears to Jenkins setup.
nuclearsandwich Mar 14, 2023
b67f9de
Only download and install the package if it is not already installed.
nuclearsandwich Mar 14, 2023
1a4aac5
Add launchd service for running Jenkins.
nuclearsandwich Mar 15, 2023
c04aee0
Drop group ownership.
nuclearsandwich Mar 15, 2023
8b6b1b1
Use launchd resource to create and start jenkins-agent.
nuclearsandwich Mar 16, 2023
9f91eb6
Copy labels array so it can be mutated.
nuclearsandwich Mar 16, 2023
f82ba3f
Use log directory within jenkins user directory.
nuclearsandwich Mar 16, 2023
2be05e2
Create jenkins-agent directory and move workspace root there.
nuclearsandwich Mar 16, 2023
0b0f02c
Specify correct group for swarm client ownership.
nuclearsandwich Mar 16, 2023
b1fd3e4
Enhance description and add architecture label.
nuclearsandwich Mar 16, 2023
9106075
Put argument flag and value on same line.
nuclearsandwich Mar 16, 2023
7d52b15
Add environment variables to agent.
nuclearsandwich Mar 16, 2023
7e8df14
Install homebrew cask and run it at Jenkins login.
nuclearsandwich Mar 16, 2023
096c41e
Install xquartz via direct download rather than homebrew cask.
nuclearsandwich Mar 16, 2023
ece1c43
One -e is required per variable.
nuclearsandwich Mar 16, 2023
9f329b9
Fix XQuartz program path.
nuclearsandwich Mar 16, 2023
9ee2d25
Document manual steps which are not yet automated.
nuclearsandwich Apr 5, 2023
55320fa
add sonoma and sequoia support
claraberendsen Sep 20, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
155 changes: 155 additions & 0 deletions recipes/macos.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,155 @@
## Underautomated setup instructions
# This section lists operations which were resistant to initial attempts to
# automate them.
# Hopefully with time, effort, and documentation we can find ways to automate these steps as well.

# Start by checking for updates and running any pending OS updates.
# Do not do major macOS version upgrades, such as Monterey -> Ventura.

# `administrator` user should already exist and the password is available in Bitwarden.

# Create `jenkins` account with sudo / administrator access to the host.
# Set it up for passwordless sudo.

# Log in as the Jenkins user, leaving accessibility, siri, and apple ID sign in disabled during initial user setup.

# Enable autologin for Jenkins from Login options, this is required so that xquartz is started on system boot.

# Verify SSH and VNC remote access are enabled, which should already true for
# our hosted machines.
# Verify remote management is enabled _only_ for administrator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# Verify remote management is enabled _only_ for administrator

This can lead to VNC connection problems


# Verify wifi and bluetooth are disabled, which should already be true for
# our hosted machines.

# Disable Spotlight indexing. It's worth doing globally but at the very least
nuclearsandwich marked this conversation as resolved.
Show resolved Hide resolved
# make sure that `/Users/jenkins` and `/usr/local` are disabled.

# In Energy Saver settings, verify that display and system sleep are disabled
# completely by setting them to Never. There is no attached display so this
# will not spend extra watts.


# Run `git` or `cc` so that macOS prompts you to install developer tools.

# Enable developer mode with `/usr/sbin/DevToolsSecurity -enable`

# As the `jenkins` user, install homebrew using the instructions on https://brew.sh

# Run `brew doctor` to verify that homebrew has no complaints post-installation.


# Install xquartz
remote_file "/tmp/xquartz.pkg" do
source "https://github.com/XQuartz/XQuartz/releases/download/XQuartz-2.8.5/XQuartz-2.8.5.pkg"
not_if "pkgutil --pkg-info org.xquartz.X11"
end
nuclearsandwich marked this conversation as resolved.
Show resolved Hide resolved

execute "install xquartz" do
command "installer -pkg /tmp/xquartz.pkg -target /"
not_if "pkgutil --pkg-info org.xquartz.X11"
end

directory "/Users/jenkins/Library/LaunchAgents" do
owner "jenkins"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick: we could abstract the user and group in the attributes file and use the variable in this file replacing the name in multiple places. Feel free to skip at this point, not sure if it worth it.

group "staff"
recursive true
end

launchd "org.xquartz.X11.plist" do
path "/Users/jenkins/Library/LaunchAgents/org.xquartz.X11.plist"
keep_alive true
run_at_load true
working_directory "/Users/jenkins"
process_type "Interactive"
program "/Applications/Utilities/XQuartz.app/Contents/MacOS/X11"
action [:create, :enable]
end
nuclearsandwich marked this conversation as resolved.
Show resolved Hide resolved


# Install java
remote_file "/tmp/jdk8.pkg" do
source "https://github.com/adoptium/temurin8-binaries/releases/download/jdk8u362-b09/OpenJDK8U-jdk_x64_mac_hotspot_8u362b09.pkg"
not_if "pkgutil --pkg-info net.temurin.8.jdk"
end

execute "install java" do
command "installer -pkg /tmp/jdk8.pkg -target /"
not_if "pkgutil --pkg-info net.temurin.8.jdk"
end
Comment on lines +70 to +79
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've opted not to prematurely add logic to support JDK8 vs JDK11, which is an upcoming transition required on the build farm. I have chosen to keep this PR straightforward and add the JDK11 support in a follow-up.


# Fetch swarm client jar
swarm_jar_path = "/Users/jenkins/swarm-client.jar"

remote_file swarm_jar_path do
source "#{node['osrfbuild']['agent']['jenkins_url']}/swarm/swarm-client.jar"
owner "jenkins"
group "staff"
end

# Map macOS platform version to version identifier
mac_version = case node["platform_version"]
when/\A11\./
"bigsur"
when /\A12\./
"monterey"
when /\A13\./
"ventura"
else
Chef::Fatal.log("macOS version #{node["platform_version"]} is not supported by this cookbook")
raise
end

agent_name = "mac-#{node["hostname"]}.#{mac_version}"
jenkins_agent_username = node['osrfbuild']['agent']['username']
jenkins_agent_user = data_bag_item('osrfbuild_jenkins_users', jenkins_agent_username)
labels = node['osrfbuild']['agent']['labels'].dup || Array.new
hw = node['hardware']
description = "macOS #{hw['operating_system_version']} #{hw['current_processor_speed']} #{hw['cpu_type']} #{hw['physical_memory']} #{} Jenkins agent"
if node['osrfbuild']['agent']['auto_generate_labels']
labels << "osx"
labels << "osx_#{mac_version}"
labels << hw['architecture']
end

directory "/Users/jenkins/log" do
owner "jenkins"
group "staff"
end


# Create workspace inside jenkins home directory
directory "/Users/jenkins/jenkins-agent" do
owner "jenkins"
group "staff"
end

launchd "org.osrfoundation.build.jenkins-agent.plist" do
path "/Library/LaunchDaemons/org.osrfoundation.build.jenkins-agent.plist"
keep_alive true
run_at_load true
username "jenkins"
working_directory "/Users/jenkins"
standard_in_path "/dev/null"
standard_out_path "/Users/jenkins/log/jenkins-agent.out.log"
standard_error_path "/Users/jenkins/log/jenkins-agent.err.log"
process_type "Interactive"
program_arguments %W[
/usr/bin/java
-jar #{swarm_jar_path}
-url #{node['osrfbuild']['agent']['jenkins_url']}
-name #{agent_name}
-username #{jenkins_agent_user['username']}
-password #{jenkins_agent_user['password']}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not familiar with Mac, but using the password argument directly here could imply a security problem if the process if visible for all users in the machine. I remember to have worked on this after a security review point that out in our infra. The linux recipe uses --passwordFile modified in #17

-description #{description}
-mode exclusive
-executors 1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick: also a good candidate to be included in an attribute file.

-fsroot /Users/jenkins/jenkins-agent
-disableClientsUniqueId
-deleteExistingClients
-labels #{labels.join(' ')}
-e HOMEWBREW_FORCE_VENDOR_RUBY=1
-e MAKE_JOBS=8
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick: we don't have a better way for defining this but maybe moving the same hardcoded value to an attribute can make people life easier if they need to modify it.

]
action [:create, :enable]
end