Background tasks

​

Pick a trigger

​

Payload jobs runner

export default buildConfig({
  jobs: {
    autoRun: [{ cron: '* * * * *', queue: 'storage-bunny' }],
  },
})

npx payload jobs:run --queue storage-bunny --handle-schedules

export default buildConfig({
  jobs: {
    access: {
      run: ({ req }) => {
        if (req.user) {
          return true
        }
        const secret = process.env.CRON_SECRET
        return Boolean(secret) && req.headers.get('authorization') === `Bearer ${secret}`
      },
    },
  },
})

{
  "crons": [{ "path": "/api/payload-jobs/run?queue=storage-bunny", "schedule": "*/5 * * * *" }]
}

​

Plugin endpoint

bunnyStorage({
  // ...
  stream: {
    // ...
    cleanup: true,
    status: { sync: true },
  },
  tasks: {
    secret: process.env.BUNNY_TASKS_SECRET, // at least 16 characters
  },
})

tasks: { secret: process.env.CRON_SECRET },

{
  "crons": [
    { "path": "/api/storage-bunny/tasks/status-sync", "schedule": "*/5 * * * *" },
    { "path": "/api/storage-bunny/tasks/cleanup", "schedule": "0 2 * * *" }
  ]
}

*/5 * * * * curl -fsS -X POST -H "Authorization: Bearer $BUNNY_TASKS_SECRET" https://your-site.com/api/storage-bunny/tasks/status-sync
0 2 * * *   curl -fsS -X POST -H "Authorization: Bearer $BUNNY_TASKS_SECRET" https://your-site.com/api/storage-bunny/tasks/cleanup

​

When hasMore is true

• the run reached its time budget of 25 seconds;
• Bunny answered 429 Too Many Requests.

for i in $(seq 1 20); do
  curl -fsS -X POST -H "Authorization: Bearer $BUNNY_TASKS_SECRET" \
    https://your-site.com/api/storage-bunny/tasks/status-sync | grep -q '"hasMore":true' || break
  sleep 5
done

​

CLI

npx payload bunny:task cleanup
npx payload bunny:task status-sync --limit 200

​

Call the functions from your own code

import { runStreamCleanup, syncStreamStatus } from '@seshuk/payload-storage-bunny'

const cleanup = await runStreamCleanup({ payload, maxDurationMs: 10_000 })
const sync = await syncStreamStatus({ payload, limit: 100, maxDurationMs: 10_000 })

console.log(cleanup.deletedCount, cleanup.errorCount, cleanup.hasMore)
console.log(sync.checked, sync.updated, sync.failed, sync.hasMore)

import config from '@payload-config'
import { syncStreamStatus } from '@seshuk/payload-storage-bunny'
import { getPayload } from 'payload'

export const GET = async (request: Request) => {
  const secret = process.env.CRON_SECRET
  if (!secret || request.headers.get('authorization') !== `Bearer ${secret}`) {
    return new Response('Unauthorized', { status: 401 })
  }
  const payload = await getPayload({ config })
  return Response.json(await syncStreamStatus({ payload }))
}

import config from '@payload-config'
import { syncStreamStatus } from '@seshuk/payload-storage-bunny'
import { getPayload } from 'payload'

const payload = await getPayload({ config })
let result = await syncStreamStatus({ payload })
while (result.hasMore) {
  await new Promise((resolve) => setTimeout(resolve, 5000))
  result = await syncStreamStatus({ payload })
}
process.exit(0)

npx payload run scripts/sync-stream-status.ts

​

Status sync

stream: {
  // ...
  status: { sync: true },
}

stream: {
  // ...
  status: {
    sync: {
      limit: 50,
      maxAge: 604800,
      schedule: { cron: '*/5 * * * *', queue: 'storage-bunny' },
    },
  },
}

1. For each collection with status sync, it finds two kinds of documents, last saved first, limit at a time, until none are left or the run stops: documents whose stored status is processing and that were saved in the last maxAge seconds, and documents with a Stream video but no stored status, at any age. Videos uploaded before the status existed have no status, so the sync fills it for them.
2. It asks the Bunny API for each video, four at a time.
3. When the state differs from the stored one, it stores the new state. A video with no stored status gets processing while Bunny still encodes it. The write happens only if the stored status has not changed since the query and the document still points at the same video, so it never overwrites a newer state from the webhook or the state of a replaced video. It keeps the stored MP4 resolutions. When the video becomes ready, Bunny has finished encoding it, mp4Fallback is on and the document has no resolutions yet, it fetches them and stores them in the same write. If that fetch fails or Bunny returns no MP4 resolutions yet, it does not store ready. The document stays or becomes processing, and a later run tries again while the document was saved in the last maxAge seconds. If Bunny answers that fetch with 429, the run stops as described below and the document keeps its stored status for the next run. updated counts only the writes that changed a document.
4. A video that Bunny no longer has becomes failed, but only when the library itself still answers. Other errors count in failed, and the document keeps its stored status for the next run. So a video with no status is checked again on every run until Bunny answers for it. A call that the run deadline cuts off does not count in failed.
5. In a collection with versions.drafts, it then does the same for the latest version of each document, with the same filters, page size and limits. The updatedAt of the version counts for maxAge. It skips a version whose document is in the trash. A version with the same video as its main document does not need a Bunny call, because step 1 already checks that video. When the main document has another state, the sync copies the state and the resolutions of the main document into the version. This copy counts in updated but not in checked. After a copy to ready or failed, the version no longer matches, and a read with draft: true does not show an old processing state. A version with another video than its main document is checked like a document. So a video that only a draft has also gets its state. The writes change only bunnyData.stream of the version and keep its other fields and its updatedAt. checked, updated and failed count documents and versions together.

​

Turn the status off

​

Troubleshooting