GeeCoolGuest
GeeCoolThe CPU side will not be the bottleneck. Between GPU generations, 20 cores/28 threads finish prefill, offload layers and embedding chores with room left over, VNNI accelerates the INT8 stretches, and a 5.6GHz boost trims single-request waits. The real wall sits elsewhere: VRAM decides how large a model this rig can hold.
With a mid-range card running 7B-class work, the pairing is smooth and balanced. If 70B-class is the target, neither core count nor CPU tier moves the needle — VRAM plus memory bandwidth bind it, and 20 cores or 24 cores will not change that. This chip ships without graphics, so the card claims first place in the parts list; otherwise the machine never posts.