How Terminal resold its GPU hours based off inference demand
Overview
Terminal's mission is to bring down the cost of intelligence to 0. Intelligence is the world’s most valuable resource, and allocating it well is the most consequential problem we can solve.
Inference
Secondary market
Challenge
Inference demand follows the work week. It runs high during US business hours and falls at night, and on Saturday and Sunday it drops close to nothing.
That gap sets the price of the product. Idle weekends and weeknights get priced into weekday rates, so a Tuesday afternoon job carries the cost of an empty Saturday. Terminal treats tokens as a commodity and prices accordingly. Utilization is the main input into what Terminal can quote a client.
Sizing the reservation does not fix it. Reserve for the weekday peak and Terminal pays for the trough. Reserve for the trough and Terminal buys the peak on spot, where the rate is higher and the capacity may not be free when a client job lands.
The third option is to reserve at peak and have someone else use the trough. Doing that alone means finding a counterparty whose demand curve mirrors Terminal’s, then pricing the hours and handing over nodes every Friday.
"We do not believe in wasting any GPU cycle. Demand for inference is cyclical. It’s very high during US daytime, lower at night, and extremely low on weekends, and it’s hard to keep utilization perfect through the weekends and weeknights. So you price higher during the weekdays to make up for the loss. We want to make inference as cheap as electricity, and that means keeping utilization as high as possible. That means finding good users of those GPUs during off-peak hours."
Shubham Gupta, CEO, Terminal
Solution
Terminal made B200 nodes available for the weekend on a Friday afternoon. By that evening they had gone to a YC company building foundation models.
Terminal reserves the nodes from Ornn on a month-to-month term and lists them for weekend reservation on the platform. The hours clear at market.
Ornn runs the handover. Terminal saves what it needs before midnight on Friday, then Ornn takes the nodes and wipes them before handing them to the weekend buyer. Terminal has them back at 12:01am Monday, while Ornn credits the resale proceeds against Terminal's next monthly invoice.
Terminal describes the arrangement in either direction. It sells the weekends, or it buys only the weekdays. The nodes bill 168 hours a week, while the weekend hours come back as credit, so Terminal's effective cost is closer to 5 days.
Results
Terminal was able to monetize unused capacity on off-peak hours and keep utilization high without finding a buyer or running the handover itself. It told Ornn which hours it did not need, and Ornn priced them, placed them, and moved the nodes.
The first weekend was listed on a Friday afternoon and reserved by that evening, to a YC company building foundation models.
Ornn sourced the buyer from its own book rather than from inbound platform traffic.