The part that is real
Going from 125 Hz to 1000 Hz is a genuine improvement and one of the few settings changes you may feel the same day. At 125 Hz the mouse reports once every 8 milliseconds, so fast swipes arrive as a coarser series of jumps and input lands later. At 1000 Hz it reports every millisecond.
If a mouse is somehow stuck at 125 Hz, fixing that is worth doing immediately. It is usually one dropdown in the mouse software.
The part that is marketing
Newer mice advertise 2000, 4000 and 8000 Hz. The theoretical latency saving from 1000 Hz to 8000 Hz is under a single millisecond. Some players on very high refresh monitors report marginally smoother motion; blind tests rarely show anyone reliably telling the difference.
Those modes also cost CPU time and, on a wireless mouse, battery life. Run 1000 Hz. If your mouse offers more and you have the cycles to spare it will not hurt, but do not pay extra for it.
How it differs from frame rate and refresh rate
Three clocks get confused constantly. Polling rate is how often the mouse speaks. Frame rate is how often the game draws. Refresh rate is how often the monitor shows a new image. A 1000 Hz mouse on a 60 Hz monitor still only shows you a new picture 60 times a second, but the engine received every one of those mouse reports and used them.
It is also why two setups can feel different at identical sensitivity: the mouse and the monitor are the same, but what happens between them is not.
What to check
Set the mouse to 1000 Hz in its software, leave it, and spend your attention on the settings that matter more: acceleration off, a sane DPI, and a cm/360 you keep for months. The mouse settings guide has the full list, including what to ignore.