Install the Gameheim Perforce extension: every submit reaches your project
For the Perforce administrator of a studio that uses Gameheim. You install one signed extension on the server, once, and from then on every submitted changelist reaches the Gameheim project it belongs to, whatever tool submitted it. The guide covers the installation from Gameheim with your agent, the same steps from the command line, more projects on one server, updates and the checks to run when something does not arrive.
Submits made outside Gameheim are invisible to the project
People submit from many places: P4V, the source control panel of Unreal Editor, the p4 command line, build machines. A tool that only sees the submits it made itself misses most of the history of the project, and anything built on that history, such as tasks closed by a changelist, lags behind what is really in the depot.
The usual answer is a trigger script on the server. It works, and it brings its own chores:
- The script needs an account that can read every changelist, and its ticket or password usually ends up in a file on the server.
- If the script calls a web service while the submit runs, a slow network makes every submit slow.
- A changelist sent while the service was down is often lost, unless someone writes a queue and a retry loop.
- Each server needs its own copy of the script and its own line in the triggers table, kept up to date by hand.
How the Gameheim extension sends your changelists
The Gameheim extension is a Perforce server extension: a small Lua program that the server installs, runs and replicates itself. Perforce recommends extensions over trigger scripts for this kind of integration. The package is signed by Gameheim, so a server that only accepts signed extensions can still install it once you trust the Gameheim certificate.
Nothing runs during a submit. A background task inside the server wakes up every ten seconds, reads the changelists submitted since the last one it sent, and sends each one, oldest first, to the Gameheim projects whose depot paths it touches. One counter per project on the server remembers how far that project got, and it moves only after Gameheim has answered for it. If Gameheim cannot be reached, the changelists simply wait in Perforce and leave as soon as it answers again, in order. A project Gameheim cannot take for a while waits alone: the others go on.
| Sent for each changelist | Never sent |
|---|---|
| Number, author, date, description and stream | The content of any file |
| Whether the changelist is restricted | Files outside the depot paths of the project |
| Action, file type and depot path of up to 500 files, and how many files there are | Passwords, tickets, the activation code or the signing secret in any log |
For workspace events: the command name (such as sync), its time and the names of the workspaces Gameheim already knows for the project | The files a command touched, its arguments, or workspaces Gameheim does not know |
The extension reads as a service user of your choice, named in its configuration (ExtP4USER). The server signs it in by itself: the password of that user is not stored in any file, neither on the server nor in Gameheim. The extension only reads changelists and sets its own counters. It never changes files, changelists, users or the protections table.
For the review your security team may ask for, with every request the extension sends, every command it runs and what it writes on the server, see Gameheim Perforce extension: security and IT review.
One extension serves every Gameheim project on the server: each project is one line of its configuration, with the project's activation code and depot paths. A changelist that touches two projects is sent to both, and each project receives only its own files.
How to install the Gameheim extension on a Perforce server
The steps are done once, by the Perforce administrator. People who submit change nothing in their tools.
- Helix Core Server 2019.2 or later on Linux, or 2021.2 or later on Windows. With replicas or edge servers, install the extension on the commit server: it reaches the edges by itself, and its background task also runs on each edge (see Large servers and edge servers below).
- A Perforce user with
superaccess, to trust the certificate, install the extension and start its background task. - Outbound HTTPS (port 443) from the Perforce server to
www.gameheim.com. No inbound port is needed. - Perforce super level on the Gameheim project, to generate its activation code: whoever holds the project's key speaks for the Perforce server. A Perforce admin sees the Gameheim trigger tab but cannot generate the code.
- A ServerID on the Perforce server (
p4 serverid), so that the background task starts again with the server. - For the installation from Gameheim: your Gameheim agent 1.3.1 or later, online and connected to the project's commit server. The command line needs only
p4.
Install from Gameheim with your agent (recommended)
In Gameheim, open Project Administration > Gameheim trigger. Under Install with the agent, your own Gameheim agent runs every Perforce step for you: no files to download and no terminal. You need Perforce
superaccess on the project and an agent connected to the project's commit server (or standalone server). Each step waits for your click:- Check the server: your agent reads the server version, its role, your access, the extension in place, the Perforce licence and the edge servers. Nothing is changed.
- Service user: create a new standard user (it takes a licence seat; its password is random, never shown and needed by nobody), or choose an existing standard user, whose password is never touched. The
reviewaccess to the project's path is added if missing, through the same protections write as Perforce administration > Permissions, with its snapshot. - Gameheim certificate: compare the fingerprint your server reads with the one Gameheim publishes, and confirm. Your agent checks it again before trusting the certificate.
- Install: installs or updates the extension, sets the service user, adds the project with a new activation code that nobody has to copy, starts the background task now and at every server start, and runs the connection test. The activation is approved automatically when the fingerprint your agent read matches the key Gameheim received; the test then runs again and should read
gameheim-test: OK. - Done: for each edge server, whether its task is already running or starts at the next restart of the edge, with the command to start it now if you prefer.
Note:An update keeps the configuration and the activation of your projects: no new code. On a Perforce server running on Windows, use the extension 1.2.1 or later: 1.2.0 cannot create its keys file there and never activates.The steps below do the same from the command line, if you prefer it or your agent cannot reach the commit server.
From the command line: generate the activation code
In Gameheim, open Project Administration > Gameheim trigger and click Generate activation code. Copy the code right away: Gameheim shows it only this once and keeps only a fingerprint of it. It works once, within 30 minutes. With it the extension binds a key pair it creates on the Perforce server to your project: only the public key travels, and the extension and Gameheim each compute the same signing secret from their own keys, so the secret never travels at all, not even at the activation. Every request is signed with it, and every answer of Gameheim is signed back. A proxy that inspects TLS traffic, or anyone who changes the extension's CaFile or HttpsProxy, can read the traffic but cannot sign a request or change an answer. You paste the code when the install script asks for it.
From the same page, download the four files:
gameheim.p4-extension,gameheim-certificate.txt,install-gameheim-extension.shandinstall-gameheim-extension.ps1.Trust the Gameheim certificate
Run the command without
As a super user--yesfirst: p4 only shows the fingerprint of the certificate. Compare it with the fingerprint shown on the Gameheim trigger tab, then run it again with--yes.p4 extension --install gameheim-certificate.txt --cert p4 extension --install gameheim-certificate.txt --cert --yesCreate the service user
Create a standard user for the extension, for example
gameheim-svc, give it a password and log in once as that user to change it. At security level 3 or higher a password set by an administrator starts expired, and the extension cannot work with an expired password.Then give it
Protections tablereviewaccess to the depot paths of your Gameheim projects in the protections table (p4 protect).reviewlets it read changelists and file lists and set its counters, nothing more. Without it, the files of a changelist are hidden from the extension. On a server with replicas or edges it needs this access on the commit server too, where it also writes the key of the Refresh mark; without it Gameheim keeps working, and every Refresh reads everything again.review user gameheim-svc * //Ocean/...Optional:
Protections table, with the super optionsuperinstead ofreview. Each person sees in Gameheim only the changelists and files Perforce shows them. By default each person's Gameheim agent works this out while it runs. Withsuper, the extension works it out for every person whose Perforce user this server confirmed, also while their agent is off, and updates it within five minutes of a change to the protections or the groups. It only asks Perforce what each user may see (p4 protects -M): it still never changes files, changelists, users or the protections table. The connection test says which of the two is in use. Withsuper, a member without a Perforce user can also follow the access of a Perforce group or user you choose in Perforce administration > P4 Users; a group or user with admin or super access is refused.super user gameheim-svc * //...Run the install script
Run the script as a super user, on any machine with
Linux or macOSp4that can reach the server. It installs the extension (or updates it, keeping its configuration), sets the service user and the Gameheim address, adds the project, starts the background task now and at every server start, and runs the connection test. It asks for the activation code without showing it: paste the code from Gameheim. The code never goes on the command line, which stays in the shell history and shows in the process list; for automation it can come on standard input.
Windows (PowerShell)./install-gameheim-extension.sh --package gameheim.p4-extension \ --service-user gameheim-svc --path //Ocean/...Unblock-File .\install-gameheim-extension.ps1 .\install-gameheim-extension.ps1 -Package gameheim.p4-extension ` -ServiceUser gameheim-svc -Path //Ocean/...On Windows,
Unblock-Fileremoves the mark Windows puts on downloaded files, which the default execution policy of servers (RemoteSigned) refuses. If PowerShell still refuses the script, your execution policy is Restricted, the default on Windows desktops: runSet-ExecutionPolicy -Scope Process RemoteSignedin the same window first. It applies to that window only.A project with several roots takes several paths:
--path //Ocean/... --path //Shared/Ocean/...on Linux,-Path //Ocean/...,//Shared/Ocean/...on Windows. Paths must end with/.... If the server reaches the internet through a proxy, add--https-proxyor-HttpsProxywith its address.Important:The script warns when the server has no ServerID. The task then runs until the next restart of the server and does not start again by itself: set one withp4 serverid <name>and restart the server when convenient.Approve the activation
The first time, the connection test at the end of the script reads
gameheim-test: WAITING FOR APPROVALand prints the fingerprint of the extension's key, such as32e5-5b5b-e726-c2f9-a50c-6c8d. Copy it, open Gameheim trigger in Project Administration and paste it under Approve the activation. A Perforce super user approves it. The fingerprint is computed on the Perforce server from the key the extension holds, so it proves that Gameheim received that key and not another one.Note:Until the activation is approved, Gameheim accepts nothing from the extension, and the changelists wait on the server: none is lost. If Gameheim says the fingerprint is not the one it received, someone else presented the code first with a key of their own. Do not approve it: generate a new code and run the script again.Check the extension
The connection test runs at the end of the script, and you can run it at any time. Every line should end well, and the last one reads
Connection testgameheim-test: OK.p4 extension --run gameheim test-connectionThen submit a changelist under the project's path from any tool. Within about ten seconds Last changelist received on the Gameheim trigger tab shows its number. The first run of the task starts from the current changelist: older history is not imported.
More projects on the same server, and updates
To add another Gameheim project, run the install script again with that project's paths, and paste its activation code when asked. The other projects stay as they are, and a line with the same paths gets the new code, which is also how you put a new code in place after generating it again.
To update the extension, run the script with the new package. It moves the configuration to the new version and removes the old one. It also replaces the background task: a running task keeps the code and the configuration it started with, so it must be restarted after every update or change. The script does it for you.
If you change the configuration by hand (p4 extension --configure Gameheim::gameheim-perforce --name gameheim), run the connection test afterwards. The old task then stops sending, so that nothing leaves with the old list of projects, and the test tells you to start a new one.
Large servers and edge servers: how Gameheim keeps its queries light
The project page shows, for each person, the highest changelist their workspace is synced to. Perforce has one way to compute it, p4 changes -m1 @<workspace>, and the server reads about as many rows as there are revisions of the synced files, plus three per file. A workspace of 100,000 files with five revisions each costs about 800,000 rows; one of 2 million files with seven revisions costs about 20 million. The cost depends on the size of the workspace, so Gameheim chooses how often to ask for it.
| Mode | When Gameheim asks | What people see |
|---|---|---|
| Every minute | Small and medium workspaces, where the query takes less than a second. This is how Gameheim always worked. | The value, as always. |
| On events | With the extension installed: only after a command that can change the value (sync, flush, update, submit, reconcile, clean, unload, reload, switch, a deleted or renamed workspace) and after changes that may concern everyone (protect, group, ldapsync -g, obliterate -y). ldapsync run as a startup command leaves no event: the extension compares the members of every group every five minutes and asks then. Also once every six hours, as a safety net. | The value, and Updating for the few seconds after a sync. |
| On request | Without the extension, when the query takes a second or more three times in a row, or when a group limit refuses it: when someone opens the workspace page (if the value is older than two minutes), when they press Refresh, when a crash arrives, and once an hour. | Updated at HH:MM and a Refresh button. A crash takes the most recent value, at most one hour old. |
Gameheim moves a workspace to On request when the query takes a second or more three times in a row, so that one slow moment of the server moves nothing, and back to Every minute when it drops under 300 milliseconds. The Perforce administrators of the project get one notice the first time it happens, suggesting the extension. With the extension, a large workspace costs one query per sync instead of one per minute.
The Perforce administrators of the project can choose the behaviour in Project Administration > Gameheim trigger > Synced changelist refresh: Automatic (the default, as described above), Always on request (on request for every workspace, also with the extension: the least load on the server) or Always up to date (never on request because of cost: on events with the extension, otherwise every minute whatever the query costs). The tab also shows how many workspaces are in each mode. In every choice a query refused by a group limit stays on request.
A manual Refresh also uses the extension. From version 1.2.0, each command that can change what a workspace has synced (sync, flush, update, submit, client, reconcile, unload, reload, renameclient, clean, switch) leaves a mark in a small file of the extension's own folder on the server, at its start and at its end. Commands that can concern every workspace (such as obliterate, protect, group, stream or typemap), and commands that name another workspace, increase the Perforce key gameheim-global-seq instead. When someone presses Refresh and nothing changed their workspace since the last full reading, less than an hour before, the agent reuses that reading and the Refresh is almost immediate. Nothing is written in the depot, nothing travels on the network, previews such as sync -n leave no mark, and the extension never refuses or delays a command: we measured about 1 to 3 ms per command on a standard server and about 4 ms on an edge. The Refresh reads everything again, as before, on a forwarding replica, on a workspace owned by another Perforce user, after the agent restarts, or with an earlier version of the extension.
Gameheim runs at most four of these computations at once on each Perforce server, the others wait their turn, so that a protections change or the morning syncs never reach the server as one burst. The limit counts the whole server, every project of your organization on it, and the Perforce administrators can change it in the same tab, under Computations at once on the Perforce server: 1, 2, 4 or 8. It is a trade-off. While a computation reads the synced files of a workspace, a sync or submit on that server may wait up to about a second; in our tests on a 2 000 000-file workspace with four computations always running, one of those commands in twenty took about 1.5 seconds instead of tens of milliseconds. Fewer at once means fewer cores and fewer waits, but the value arrives later when many people sync together. A server counts once the Gameheim extension has reported its ServerID; on a server without the extension, or without a ServerID, it stays four at once per project: install the extension, and give the server a ServerID with p4 serverid if it has none, to choose.
On edge servers the background task also runs, started by the same startup.N at their next restart, or at once with the command the install script prints for each edge. There it only hands the workspace events of the edge's users to the commit server, in Perforce keys gameheim-ws.<ServerID>.*, and marks itself alive in gameheim-ws-alive.<ServerID>. The commit server sends them to Gameheim with the changelists. Until the task runs on an edge, the workspaces on that edge stay on Every minute or On request: nothing is missed, it only costs more.
Group limits. The query runs with the Perforce user of each person, so the MaxScanRows, MaxResults and MaxLockTime of their groups apply. When a limit refuses it, the page says why, the workspace moves to On request and Gameheim tries again once an hour instead of every minute. To have the value for very large workspaces, raise MaxScanRows for the groups of those people above the rows estimated above, or leave the limit as it is: the rest of Gameheim works without the value.
Configuration reference
| Where | Field | Value |
|---|---|---|
| Global configuration | ExtP4USER | The service user the extension reads as |
| Global configuration | GameheimUrl | https://www.gameheim.com: the only address the extension talks to, written in its signed code |
| Global configuration | HttpsProxy | Proxy address, or - for none |
| Global configuration | CaFile | CA bundle of the machine, or - to let the extension find it |
| Global configuration | ServerFingerprint | The SSL fingerprint of this server, written by the Gameheim wizard (1.2.3): the background task trusts only this certificate on its own connection. - with the script: trusted at the first connection |
Instance gameheim | Projects | One line per project: <activation code> //depot/path/..., more paths separated by spaces |
| Server | startup.N | bgtask -t gameheim -i 10 -m 2000000000 -b 2000000000 |
The instance must be called gameheim: the Gameheim agent runs p4 extension --run gameheim to confirm identities. The extension keeps these counters on the server: one gameheim-feed.<id> per project (the last changelist sent to it; the id is a fingerprint of the activation code, never the code), gameheim-feed-owner, gameheim-feed-config and gameheim-feed-heartbeat. A server that ran an earlier version also has gameheim-feed: each project starts from it, then the extension deletes it. On a server with edges the extension also keeps the keys gameheim-ws.<ServerID>.* (workspace events waiting to leave, deleted once sent) and gameheim-ws-alive.<ServerID> (when each edge task last ran); the service user needs review access to read and delete them. The counter gameheim-ws-groups holds a fingerprint of who is in which group (p4 groups, members and subgroups only), compared every five minutes. The keys of the activated projects are in the file gameheim-keys, in the extension's folder under server.extensions.dir on the Perforce server: beside the folders of each installed version, so that an update keeps them. Only the account p4d runs as can read or write it, and no p4 command reads it. From version 1.2.0 the extension also keeps the key gameheim-global-seq (increased by the commands that can concern every workspace) and, in the same folder, one file ws-<16 hexadecimal characters> per workspace with the marks of its commands, one run-… file for each command still running (deleted when it ends) and a marker-epoch file. They hold random marks, times and the process id of a running command: no workspace names, paths, users or command arguments.
Troubleshooting: changelists do not reach Gameheim
To see what the extension knows about a workspace, its owner can run p4 extension --run gameheim state <workspace>. It prints one line starting with gameheim-state:, with the workspace's mark, the key, the server and whether the extension can write its files and the key (keys=ok, no or error). Anybody but the workspace's owner gets gameheim-state: denied.
| What you see | What to do |
|---|---|
| Installation failure: extension package is signed by untrusted certificate | Trust the Gameheim certificate first, then run the script again. |
| Gameheim does not recognise this activation code, or it expired | The code works once, within 30 minutes. Generate a new one in Gameheim trigger and run the script again with it. |
| This activation code was already used, or someone else presented it | The code works once, for one Perforce server. Generate a new one in Gameheim trigger and run the script again with it. |
| Gameheim refused the signature of this server | Check that the server's clock is right (within 5 minutes). If the code was generated again in Gameheim, run the script again with the new one. |
| An answer arrived without Gameheim's signature | Something between the server and Gameheim answered in its place, or changed the answer: the extension ignored it and will try again. Check the proxy in HttpsProxy and the certificates in CaFile. |
| No secure random source | The extension creates its key from /dev/urandom on Linux and from PowerShell on Windows. On Windows, allow the Perforce service to run %SystemRoot%\System32\WindowsPowerShell\v1.0\powershell.exe, then run the connection test again. |
| The extension's data directory is not where this version expects it | The extension keeps no key and does not activate. Send the output of p4 extension --list --type extensions to Gameheim support. |
| Notice: the Gameheim extension connects differently | Its CaFile or HttpsProxy changed. If nobody in your team changed them, find out who did. Since admin users can configure signed extensions unless server.extensions.allow.admin is 0, consider p4 configure set server.extensions.allow.admin=0. |
| Gameheim could not be reached | Allow outbound HTTPS to www.gameheim.com, by host name or with the AzureFrontDoor.Frontend service tag, or set HttpsProxy. |
| The server could not verify Gameheim's certificate | On Linux distributions that keep certificates in an unusual place, set CaFile to the CA bundle of the machine. |
| Service user cannot run Perforce commands | Its password has expired or was never set. In Project Administration > P4 Users, a Perforce super user can choose Set password > Nobody knows it for that user; or log in once as that user and change the password. |
| Cannot set counters | Give the service user review access in the protections table. |
| Background task: not running | Start it with p4 bgtask -t gameheim -i 10 -m 2000000000 -b 2000000000 -d. If it stops at every restart, set a ServerID. |
| Background task: stopped, because this server's SSL certificate changed | The certificate was renewed, or changed. If you expected it, run the installation again from Project Administration > Gameheim trigger: Gameheim reads the new certificate through your agent and gives it to the extension. If nobody renewed it, find out who changed it first. |
| Background task: running, but each run fails | The line carries the first line of the error. A Perforce error names the cause (protections, password, limits); send the line to Gameheim support otherwise. |
| Background task: running an older version or configuration | Start a new task with the same command: the old one stops sending by itself. |
| Workspaces on an edge still refresh every minute, or on request | Start the task on that edge: p4 -p <edge address> bgtask -t gameheim -i 10 -m 2000000000 -b 2000000000 -d, or restart the edge. Check that it has a ServerID. |
| The workspace page says the server limits refuse the query | A group limit of that person (MaxScanRows, MaxResults or MaxLockTime) refuses p4 changes -m1 @<workspace>. Raise it for their groups, or leave it: only that value is missing. |
| Notice: the Gameheim extension did not send some changelists | Run p4 extension --run gameheim test-connection on the server. If it says OK, check that the depot paths of the project in the extension configuration cover every stream the team submits to. |
Remove the extension
Unset the startup command the script added, stop the task (its process id is in p4 monitor show -al, on the commit server and on each edge), delete the extension, its counters and its keys:
p4 configure unset startup.<N>
p4 monitor terminate <process id>
p4 extension --delete Gameheim::gameheim-perforce --yes
p4 counters -e "gameheim-feed.*"
p4 counter -d gameheim-feed.<id>
p4 counter -d gameheim-feed-owner
p4 counter -d gameheim-feed-config
p4 counter -d gameheim-feed-heartbeat
p4 counter -d gameheim-ws-groups
p4 key -d gameheim-global-seq
p4 keys -e "gameheim-ws*"
p4 key -d <each key listed>Frequently asked questions
Does a submit wait for Gameheim?
No. The extension does nothing while a submit runs. A background task on the server reads the submitted changelists every ten seconds and sends them afterwards, so a slow network or an outage never slows down P4V, Unreal Editor or the command line.
What happens to changelists while Gameheim cannot be reached?
They wait in Perforce. The extension moves a project's counter only after Gameheim has stored the changelist for that project, so when Gameheim answers again the waiting changelists leave in the order they were submitted.
Does Gameheim receive the content of our files?
No. For each changelist it receives the number, author, date, description, stream, whether it is restricted, and the action, type and depot path of up to 500 files under the project's paths. File content never leaves the server.
Can one Perforce server feed several Gameheim projects?
Yes. One extension serves them all: each project is a line with its activation code and depot paths. Run the install script once per project. A changelist that touches two projects reaches both, each with its own files only.
How much load does Gameheim put on a large Perforce server?
Locks cost about two rows per opened file, read once a minute while someone uses the PC and every five minutes once it is idle. The synced changelist is the expensive query on large workspaces: with the extension Gameheim computes it only after a sync or a change of protections, and without it, on large workspaces, only when someone looks. See Large servers and edge servers above.
Where does the password of the service user live?
Nowhere outside Perforce. The server signs the extension in as the service user by itself, so the password is not written in any file on the server and Gameheim never sees it.
