How exactly are women a scarce resource?

They are not. As you stated, there are plenty of them. It is sexual access that is scarce, not women.

No doubt there is a good deal of sex occurring in the world. In the last few decades, at least 7.5 billion instances are known to have occurred

The term “scarcity” refers to the difference between the amount desired and the amount available. It also refers to the amount of effort required to obtain an adequate supply and considers the portion of the population that is able to acquire an adequate supply compared to the portion that is not.

“Is it because most women won’t have sex with just anyone?”

That could be a contributing factor.

“skeevy guys who really don’t have much to offer get laid.”

Skeevy guys who aggressively pursue sexual access, who devise clever strategies and carry them out effectively, who achieve wealth and power that can be exchanged for sexual access, and who outdo competitors get laid, but not necessarily as frequently as they would like.

One woman’s “skeevy guy” may be another woman’s “heartthrob.”

“… don’t have much to offer...”

As you have observed, guys are expected to have “something to offer” (besides sex itself). The expectation that one must offer something in exchange for a resource is a characteristic of scarcity.