A private GPU server sits beside law books and brass scales on a desk

Lawyer Builds Private AI Server: Eight Used V100 GPUs, Zero Cloud

Key Takeaways

  • A lawyer builds private ai server hardware from eight used 2018-era Nvidia Tesla V100 GPUs so that case files never leave the office and no client data touches a cloud service, according to reporting by Tom's Hardware, covered by TechJuice on 10 November 2026.
  • The server runs DeepSeek V4.1 Flash, a 552-billion-parameter open-weight model, and the builder measured up to 215 tokens per second on code and 118 tokens per second on legal writing — his own measurements, not an independent benchmark.
  • The two four-GPU boards cost about $2,400 each when bought, but that is not the whole build: the system also needs a Threadripper Pro platform, 256GB of RAM and 2,800W of power across two office circuits, and the builder has not disclosed a total cost.
  • The lawyer estimates that 50% to 60% of his existing AI workflows now run locally instead of on frontier cloud models, handling first-pass case-file review, summaries and routine drafts — always with human review before anything leaves the office.
  • The catch is time: Nvidia's R580 driver branch is the last to support the Volta architecture in the V100, and used V100 prices on eBay have already risen from about $400 to about $900 per card.

A civil litigation attorney had a problem that no cloud subscription could solve. Every time he pasted a case file into a hosted AI tool, privileged client information left his office and entered someone else's system, under someone else's terms. So he built his own answer: a private AI server assembled from eight used Nvidia Tesla V100 graphics cards from 2018, running an open-weight model entirely on hardware he owns. The story surfaced in a Reddit post on r/LocalLLaMA in early November 2026, was reported by Tom's Hardware on 9 November, and was picked up by TechJuice on 10 November 2026 — the week the query "lawyer builds private ai server" broke out in search.

What makes the story worth more than a day of attention is not the novelty of running a model locally. Hobbyists have done that for years. It is that this machine does real, daily legal work at speeds that look like a production system, on hardware that data centres retired years ago, for a fraction of what new AI hardware costs. It is also a story with honest limits: the builder's own numbers, a single-request system, a burnt power connector, and a driver-support clock that is already ticking. This article sticks to what the reporting actually says, separates the builder's measurements from verified facts, and explains why a lawyer — of all professionals — would go to this much trouble.

Who built it, and why a lawyer builds a private AI server

The builder is a civil litigation attorney who described the project on Reddit, in the r/LocalLLaMA community, where people run large language models on their own machines. Secondary coverage identifies him as Nick Mermiges of PLI Labs, and links a public repository documenting the build, including roughly thirty optimisations that did not work. The original post framed the project plainly: the entire point is privacy. Case files never leave the office, and no client data touches a cloud service.

That framing matters because, for a lawyer, confidentiality is not a preference. It is a professional duty. The American Bar Association's Model Rule 1.6(c) requires a lawyer to "make reasonable efforts to prevent the inadvertent or unauthorized disclosure of, or unauthorized access to, information relating to the representation of a client." The ABA's Formal Opinion 512, issued on 29 July 2024, applied that duty — along with competence, communication and supervision — to generative AI, and warned lawyers to understand how a tool handles data before client information goes into it: whether inputs are used for training, who can access stored prompts, how long data is retained, and what the vendor's terms actually permit. A system that keeps every prompt, document and output on hardware in the lawyer's own office answers those questions by architecture rather than by contract. Nothing is disclosed to a vendor in the first place, so there is nothing to leak, retain or train on.

That does not make a local server automatically compliant with every ethics rule — a lawyer still has to supervise the output, verify citations and protect the machine itself — but it removes the single largest confidentiality exposure in cloud AI use: the transfer of privileged material to a third party. It is the same logic that has pushed some firms toward private deployments generally, and it explains why this particular build resonated far beyond the homelab community. The risk it addresses is one every litigator recognises. Our earlier look at AI hallucinations in legal filings covers the other half of the problem: even when the data stays private, the output still has to be checked, because models invent citations and get facts wrong.

The hardware: eight retired data-centre GPUs

The heart of the machine is eight used Nvidia Tesla V100-SXM2-32GB GPUs, a data-centre card from 2018 built on Nvidia's Volta architecture. Each card carries 32GB of HBM2 memory running at 900 GB/s of bandwidth. The eight cards are split across two four-GPU SXM2 boards, each bought for roughly $2,400 — about $4,800 for the boards combined. Inside each board, all four cards communicate over NVLink, giving 128GB of combined GPU memory per board. The two boards, however, only talk to each other over the slower PCIe bus, and that single detail shaped the whole design.

The rest of the system is built around a Gigabyte Threadripper Pro motherboard with a 12-core AMD 3945WX processor, 256GB of DDR4 RDIMM system memory, and two power supplies delivering a combined 2,800W across two regular office circuits. Nothing about that list is exotic by data-centre standards, and that is the point. The GPUs are retired hardware, the CPU is a workstation chip, and the power comes from wall sockets rather than a raised-floor data centre. The builder did not buy a server. He assembled one from parts that the AI industry had already moved past.

It is important not to misread the price. The $2,400-per-board figure is what the lawyer paid for the two GPU boards. It is not the total cost of the server. The motherboard, processor, memory, power supplies, PCIe switches and cabling all cost money, and the builder has not disclosed a total. Reporting that calls this a "$4,800 AI server" would be wrong. What can fairly be said is that the most expensive component — the compute — cost a small fraction of what equivalent new hardware would cost, and that used V100 prices have moved sharply since: cards that were about $400 each when the original post went up were listed on eBay at about $900 each by the time the story was reported, more than doubling. Anyone pricing the same build today would pay considerably more for the GPUs alone.

The model: DeepSeek V4.1 Flash, squeezed to fit

The server runs DeepSeek V4.1 Flash, an open-weight model released by DeepSeek on 10 September 2026 under the MIT licence, with weights published on Hugging Face. It is a 552-billion-parameter mixture-of-experts model, but only about 8 billion parameters are active per token during input processing and about 16 billion during output — the sparsity that makes the "Flash" line cheap to run. It supports a context window of up to one million tokens, and an updated checkpoint, V4.1-Flash-1017, followed on 17 October 2026. The MIT licence is what makes this kind of project legally straightforward: the weights can be downloaded, run commercially and kept entirely inside private infrastructure. DeepSeek itself has been raising capital at state-backed scale, as covered in our report on DeepSeek's $15 billion second round — which makes the contrast sharp: the company behind the model is raising billions, while one lawyer runs that same model on retired hardware.

Fitting a 552-billion-parameter model onto eight 32GB cards is the engineering feat at the centre of the story. The builder compressed the model to mostly 2-bit precision, bringing the weights to a reported 248.8GB — just inside the 256GB of combined GPU memory across the eight cards. The model's expert layers sit entirely in GPU memory, while about 110GB of Engram lookup tables reside in system RAM. Fetching from RAM adds roughly 0.12 milliseconds per step, which the reporting says barely affects speed. The post title originally claimed every weight was in VRAM; an edit corrected that, since the Engram tables sit in system RAM. That correction is worth noting because it shows the numbers were being checked and revised in public, not polished for an audience.

How the model is split across the two boards turned out to be critical. The first 20 layers run on one board and the second 20 on the other, so each board handles its heavy traffic over its own fast NVLink connection and only one handoff per step crosses the slower PCIe link between boards. An earlier attempt to distribute every layer across all eight cards caused speeds to collapse, because every step then had to cross that PCIe bottleneck. The fix was architectural rather than financial: arrange the work so the slow link is crossed as rarely as possible. It is a neat illustration of why local AI is as much a systems problem as a shopping problem — the same eight cards perform very differently depending on how the model is laid across them.

The performance numbers — and what they actually mean

The builder's measurements, taken on 5 November 2026 using greedy decoding with thinking disabled and 1,024 output tokens, averaged across two runs, were: up to 215 tokens per second for code generation, 118 tokens per second for legal writing, and 89 tokens per second for prose. Prefill — the speed at which the model reads a prompt — reached about 2,200 tokens per second at a 128,000-token context, and the system supports up to a 524,000-token context per request. At a 128,000-token prompt, the first response takes about a minute.

Those numbers need careful handling, because they are the builder's own measurements and not an independent benchmark. No third party has reproduced them, the test conditions were chosen by the builder, and tokens per second is a speed measure, not a quality measure. A fast model can still produce a wrong summary, a bad citation or a misleading draft. What the numbers do establish is that the system is fast enough for interactive daily work: at 118 tokens per second, a 1,000-token legal draft takes under nine seconds to generate once the model has read the prompt. The bottleneck is not generation speed but the time to read a long case file, which is why the roughly one-minute first-token delay on very long prompts matters more in practice than the headline decode rate.

For comparison, the speeds are in the same range as a good cloud API for a single user — and that is the relevant comparison, because this is a single-user system. It processes one request at a time. It cannot serve several lawyers or several jobs in parallel, and multi-request concurrency is unfinished work. A firm imagining this as a shared resource for twenty attorneys would be misunderstanding the build. What it is, right now, is one lawyer's private assistant, fast enough that he does not have to wait for it.

What it does all day

The server handles daily first-pass legal work. It reads folders of case records, extracts facts, dates and figures, generates summaries, and drafts routine documents. Retrieval databases and tool calls connect through a Discord-style interface over SSH from a Windows PC, so the lawyer talks to the system from his desk while the server sits elsewhere in the office. Nothing goes out without human review. The model does the first pass; the lawyer does the reading, the judgement and the sign-off.

The lawyer estimates that 50% to 60% of his existing AI workflows now skip frontier models entirely and run on this machine. The rest runs on a second DeepSeek server built from two Nvidia DGX Spark systems, with the two machines handing work to each other. That second system is a reminder that the V100 server is not the whole story: the builder has more than one local machine, and the workload is split between them. But the V100 build carries the majority share, which is striking for hardware that was new when the first Trump administration was still in office.

The builder has described the project as mostly a hobby for now, with a possible business idea in the future and no timeline offered. That framing matters for readers tempted to treat this as a product announcement. It is not a product. There is no price list, no support contract and no promise that the setup will work for anyone else. It is a working experiment by one technically capable lawyer, documented in public, that happens to answer a question many firms are asking privately: can capable AI run locally, on affordable hardware, without sending client files to a cloud provider? The answer this build gives is a qualified yes — qualified by the effort it took to get there.

The hard parts: crashes, a burnt connector and memory errors

Getting the system stable required real troubleshooting, and the reporting is refreshingly specific about it. The host crashed twice from fatal PCIe errors when GPU clocks ran uncapped. At full power, cards drew up to 232W each, well past their 180W limit. One board's power connector showed a visible burn mark. The lawyer removed that board and capped every remaining card at 150W with clocks limited to 1,522 MHz. Separately, the system RAM hit uncorrected ECC errors until its speed was dropped from DDR4-3200 to DDR4-2666.

None of that is a reason to dismiss the build. It is the reason to take it seriously. Used data-centre hardware pushed hard in an office, on regular circuits, with second-hand power delivery, will find its limits — and this builder found them, diagnosed them and worked around them by accepting lower clocks and lower power in exchange for stability. The capped configuration is the one producing the 215 and 118 tokens-per-second figures, not the uncapped configuration that crashed. That distinction is exactly the kind of detail that gets lost when a story is retold as "lawyer gets 215 tokens per second from old GPUs", and it is why the original reporting deserves reading in full.

It is also a warning for anyone considering a copy. This project took, by one account of the build, more than four months of work, including recompiled Linux kernels, CUDA troubleshooting and custom inference engineering, with help from Claude Code to finish the job. A non-technical lawyer cannot reproduce it by ordering the same parts. The hardware is the easy half. The software — the quantisation, the layer split, the kernel work, the retrieval setup — is where the months went. Firms that want the privacy without the engineering project are better served by the growing market of managed private deployments than by a parts list.

The clock running out: drivers and prices

There is one looming risk, and it is not performance. Nvidia's official compatibility matrix lists R580 as the last driver branch supporting the Volta architecture, and CUDA 12.x as its last supported toolkit family. Once Nvidia moves on, the V100 loses official software support: no new driver features, and eventually no compatibility with newer software stacks that assume newer CUDA versions. The reporting around the vLLM inference engine makes the consequence concrete — its November 2026 release moved to CUDA 13 only, dropping Volta support, so a V100 system must stay pinned to older releases or replace the hardware. The machine keeps working on the day support ends, but every month after that, the software around it moves a little further away.

The second pressure is price. Used V100 cards that cost about $400 each when the original post went up were listed at about $900 each within days, and listings suggested prices were still rising. Attention itself is bidding up the hardware: every article about this build sends more buyers to the same small pool of retired cards. The window in which eight used V100s are a bargain is narrowing from both directions at once — the hardware is getting more expensive while its supported software life is getting shorter. That does not make the build a mistake. It makes it a moment in time, and a lawyer starting the same project in six months may find a different answer to the same question.

What this means for law firms — and what it does not

For the legal profession broadly, the build demonstrates something important: running capable AI models locally is no longer limited to technology companies with data-centre budgets. A lawyer with used hardware and two office circuits built a system that handles real case work at production speeds. In a profession where client confidentiality is not optional, that matters more than any benchmark — and it arrives at a moment when the profession's own tools are moving in the opposite direction. Cloud legal AI is consolidating fast: Harvey, the legal AI company, recently acquired Aida, as covered in our report on the Harvey buys Aida legal AI acquisition, and the major legal research platforms are all cloud-hosted. The V100 server is the counter-argument made in hardware: the cloud is a choice, not a law of nature.

But the counter-argument has limits that should be stated plainly. First, a local model is only as good as the model. DeepSeek V4.1 Flash is a strong open-weight model, but it is not a frontier model, and the builder's own estimate concedes that 40% to 50% of his workflows still need something else — in his case, a second local server, not a cloud API. A firm whose work needs the strongest available reasoning may find that a local model, however fast, is not enough. Second, privacy is not the same as accuracy. A model running in your office can still hallucinate a case citation, and the duty to verify output does not shrink because the server is yours. Third, security cuts both ways: a cloud provider's security team is replaced by whoever maintains the office server, and an unpatched local system holding every case file in the firm is a risk of its own. "No data leaves the office" is a strong claim only if the office itself is secure.

The practical lesson is narrower and more useful than either the hype or the scepticism. For a solo practitioner or a small firm with one technically capable person, a local server can now carry the bulk of first-pass document work — extraction, summarisation, routine drafting — without any client data leaving the building, at a hardware cost measured in thousands rather than hundreds of thousands. For a larger firm, the same logic points toward managed private deployments and on-premise options from legal AI vendors, not toward asking every partner to build a server. And for any firm, the decision should start from the ethics rules, not the hardware catalogue: understand what the tool does with data, confirm the output, and keep a human in charge of anything that leaves the office.

The bottom line

A lawyer builds private ai server hardware out of eight used V100 GPUs because confidentiality demanded it, and the result is a machine that does real legal work at 118 tokens per second on legal writing, keeps every case file in the office, and carries half or more of its builder's AI workload. The build cost less than new hardware would have, took months of engineering, survived crashes and a burnt connector, and now faces a finite software-support window as Nvidia retires Volta and used prices climb. It is not a product, and it is not a template every firm can copy. It is proof that private AI is no longer a data-centre privilege — and a reminder that the hard part was never buying the GPUs. It was everything after that.

Sources

Similar Posts