Troubleshooting
Symptoms first, then the checks in the order that finds the cause fastest. Every agent also ships its own log to the console, so most of this can be diagnosed without touching the machine.
Where the logs are #
| What | Where |
|---|---|
| A job run: what it did, file by file | Console → Jobs → ⋮ → Run History → the run. Failed files are listed with the reason for each. |
| The agent's own log, from the console | Shipped continuously (INFO and above). Readable with the get_agent_logs tool, or ask us. |
| Windows, on the machine | C:\ProgramData\Aurawolf\logs\agent.log (rolling daily); the tray writes tray.log beside it |
| macOS | sudo launchctl print system/com.caryvane.agent; log files under /Library/Application Support/Caryvane/logs/ |
| Linux | journalctl -u caryvane-agent |
Agent shows Offline #
An agent is Online while it has checked in within the last 90 seconds. Offline means the console has not heard from it; the machine may simply be off — laptops and historic machines are expected to come and go, and that is not a fault. If it should be on:
- Is the service running? Windows:
Get-Service CaryvaneAgent. macOS:launchctl print system/com.caryvane.agent. Linux:systemctl status caryvane-agent. - Can it reach the console? From the machine:
curl -sI https://console.caryvane.com/healthz(or open it in a browser). Outbound 443 only; a proxy that inspects TLS must trust the certificate chain or exempt the host. - Is it mid-update? The agent stops for a minute when a new version installs; it comes back on its own. The version column shows whether it did.
- Read the log. The last lines of
agent.logsay what it was doing. "Server rejected the machine token" repeating means the enrolment is dead.
Agent installed but not in the console — or the tray says Registered and the console disagrees #
Three causes, in order of likelihood:
- Not enrolled. A silent install with a token that had expired or been used leaves the agent installed and waiting. Right-click the tray icon → Enroll Agent… with a fresh token. On macOS/Linux, write
setup.json/agent.jsonwith the token and restart the service (Deploy the agent). - Enrolled somewhere else. The machine holds an enrolment from a different server (a pilot environment) or a customer that was deleted, and the console it is pointed at rejects it. The current agent detects this, discards the dead enrolment and asks for a new token from the tray. If a machine is stuck like this on an old build, install the current version over the top — no uninstall needed — then enrol.
- Wrong customer. The token decides the customer. Look under the other customers you can see; if it enrolled into the wrong one, uninstall (which forgets the enrolment) and reinstall with the right token.
The installer and every binary inside it are code-signed. A prompt on an old download usually means an unsigned pre-release; download the current installer from the console.
Job finished with a Warning #
Warning means the run completed and some files were skipped. The run log names every one with the reason. The common ones:
| Log line | Cause | What to do |
|---|---|---|
SKIPPED (encrypted with EFS — the agent cannot read it) | Windows EFS-encrypted files are readable only by the user who encrypted them; the agent runs as SYSTEM. | Decrypt them (cipher /d), or exclude the folder. This is a warning by design, not a failure. |
Skipped (in use by another program) | An exclusive lock, and VSS was off or could not snapshot the volume. | Turn on Use VSS for the job; for databases and VMs use an App-consistent job. |
… contains a virus or potentially unwanted software | Windows Defender blocked the read. Often an old keygen or crack in a user's Downloads. | Exclude it, or let the customer delete it. Caryvane will not bypass the antivirus. |
Skipped (stored in the cloud, not on this machine) | A OneDrive/Dropbox placeholder — the file's bytes are not on the disk. | Back up the cloud account with a connector or storage-target job instead. |
Job Failed #
- "Interrupted: the agent restarted while this run was in progress." The machine or the service restarted mid-run (Windows Update at 3 am is the classic). The next scheduled run picks up where the incremental left off. If it recurs nightly, check the machine's restart schedule against the job's.
- Storage errors (access denied, bucket not found, signature mismatch) — Storage → Test on the target. A rotated key, an expired SAS, an OAuth token revoked by the customer's admin all show here first; Egnyte tokens expire after 30 days and are refreshed automatically only while the connection is in use.
- "Source and destination must be different" / cloud-reachability errors — a Cloud Agent job cannot read a LAN share or local folder; run it on an agent instead.
- Cancelled: superseded by a newer run. A retry was queued after the failure but the next scheduled run started first. Informational.
Hyper-V jobs #
- "Production checkpoints cannot be created" for a VM — the guest cannot quiesce (Linux without the integration services, or a Windows guest with the VSS integration service off). Caryvane takes a crash-consistent checkpoint instead and says so in the log; enable Backup (volume shadow copy) under the VM's Integration Services to get application consistency.
- A checkpoint left on the VM after an interrupted run — the agent records every checkpoint it takes and merges any it finds left over at the start of the next run. If a VM shows a stale backup checkpoint days later, delete it in Hyper-V Manager; the merge happens live.
- Runs slower than expected — a full-VM backup moves the whole
.vhdx; the size in Run History is the file size, not the disk's virtual size. Incremental (changed-block) backup on Server 2016+ reduces that to what changed each night once enabled for the agent.
Cloud drive not mounting #
- Agent online and enrolled as the agent the drive was added to. Drives are pushed at check-in; an offline agent has nothing to mount.
- Windows: the Dokany driver must be installed (Add or Remove Programs → Dokan Library). The installer adds it; a first-time install on some systems needs a reboot before the driver loads. The chosen drive letter must be free.
- macOS: a user must be logged in (the FSKit module runs in the user session) and
Caryvane.appmust have Full Disk Access. Check System Settings → General → Login Items & Extensions → File System Extensions shows Caryvane enabled. - Linux:
fusermount3must exist and the mount point must be creatable; see the journal. - The storage target must test OK — a drive on a target with a dead credential mounts empty or not at all.
Connector stopped working #
- Microsoft 365: "consent" or 401 errors — a Global Administrator removed the app or its permissions. Connectors → Microsoft 365 → Connect again to re-grant consent.
- Google Workspace: 403 on a user — domain-wide delegation was changed, or the scopes list in the admin console no longer matches. Re-verify on the Connectors page names the missing scope.
- New users not being backed up — the job lists specific people. Edit it and choose all for the resource type; from then on new users are swept in automatically.
A job ran at the wrong time #
Cron schedules are in UTC. 0 2 * * * is 02:00 UTC — 03:00 during British Summer Time. Adjust the hour, and expect the wall-clock time to move by an hour at the clock change.
Still stuck #
Email hello@caryvane.com with the customer, the machine name and the run's time; we can read the agent's shipped log from our side without a screenshot. For a machine that cannot reach the console at all, attach the last 200 lines of agent.log.