Solutions · Media & entertainment

Media & entertainment

Render and transcode farms starved by storage, with peaks that double overnight.

The constraint

What makes this sector awkward

Artists and render jobs are only as fast as the storage behind them. Load is spiky and high, files are enormous, and the deadline is usually a release date rather than a quarter. The storage fabric is usually the real ceiling, and it is usually the last thing anybody suspects.

Parallel filesystem for high-bandwidth reads, a storage fabric sized for the real peak rather than the average, a scheduler that packs jobs properly, and burst capacity that does not require a new rack.

Where you probably are

Four situations we see constantly

If two or three of these describe your environment, the audit is worth the money. If all four do, it is overdue.

NFS saturates long before the network does

A single NFS server becomes the ceiling, and adding clients makes it worse. The network has headroom that the file service cannot use.

The array gets blamed for slow writes

Latency is attributed to storage hardware when the queue depth and path selection are set for a general-purpose workload rather than a sequential one.

Queue times double during crunch

The scheduler packs jobs by count rather than by resource, so a handful of jobs hold the whole cluster while producing almost nothing.

Burst capacity means a purchase order

Peak weeks are handled by buying machines that then sit idle for the rest of the year.

How we sequence it

Four phases, each useful on its own

Sequenced so that stopping after any phase leaves you better off rather than stranded mid-migration.

  1. 01

    Find the real ceiling

    Separate the file service, the fabric and the array in the measurements. Three components blamed collectively is how the wrong one gets replaced.

    You end up with

    A throughput baseline per layer, with the binding constraint identified rather than assumed.

  2. 02

    Replace the file service ceiling

    Move high-bandwidth reads onto a parallel filesystem or a properly striped layout, sized for the observed peak instead of the quarterly average.

    You end up with

    Measured read throughput against the render baseline, with the previous ceiling documented.

  3. 03

    Tune the fabric for sequential load

    Set queue depths and multipath selection for throughput rather than for general-purpose random IO, then verify against the array's own preference.

    You end up with

    ISL and path latency baselined per fabric, with path selection matched to the workload.

  4. 04

    Make scheduling match the workload

    Rework packing and priority so render jobs get the resources they claim, and interactive artist sessions do not queue behind batch work.

    You end up with

    Peak handling that does not require manual intervention or a human deciding who matters today.

What we measure in this sector

These are the numbers we put in the delivery report. If we cannot measure it, we do not promise it.

  • Measured read throughput against the render baseline
  • ISL and path latency baselined per fabric
  • Peak handled without manual intervention
  • Interactive session latency held through batch peaks

What we don't do

Every vendor says yes to everything. Here is where our answer is no, or where someone else needs to be in the room. If this list looks wrong for your situation, tell us — we would rather argue about scope before the contract than after.

  • Out of scope

    Creative application support and plugin troubleshooting

    We handle the infrastructure under the tools. Application crashes and plugin conflicts belong with the tool vendor.

  • Requires partner

    Cabling, optics and physical build-out

    We design and verify the fabric. Running cable and terminating fibre is a specialist trade we bring in rather than pretend to do.

  • Out of scope

    The content itself — media management, archive catalogues, transcoding pipelines

    We provide the storage and the throughput. Asset management software is its own discipline with its own vendors.

  • Included

    Storage architecture, fabric tuning, scheduler configuration

    Including the peak-load testing and the baselines you keep afterwards.

Questions we get asked

Media & entertainment specifics

Do we have to replace the array?

Usually not. We have found the array to be the binding constraint less often than the file service or the fabric. The audit exists to answer that question with numbers before you spend anything.

Can you work around our release calendar?

Yes, and that drives the sequencing rather than the other way round. Migrations are planned in waves that fit between deliveries, with each wave independently reversible.

Is a parallel filesystem worth it at our scale?

Depends on concurrent read bandwidth rather than capacity. If reads are concentrated and bandwidth-hungry, yes. If the workload is mostly sequential writes of independent frames, a well-tuned scale-out option is often cheaper. We will tell you which one you are.

Want this looked at against your media & entertainment environment?

Thirty minutes, no slides. Bring the awkward constraint — that is usually the interesting part.