How to test custom selenium selector? Or how to create fake WebDriver from file?

I have a custom selector which extends org.openqa.selenium.By class and implements its findElement and findElements methods. These implementations contain non-trivial CSS and logic therefore I would like to have automated tests for them. The problem is that this selector sits within a library and therefore I would like to keep these tests lightweight.

I would ideally like to create a WebDriver from static file but I cannot find any way to do this. I don't want to have a dependency on firefox/chrome driver being installed on the machine. HtmlUnitDriver should allow me to do this but it still requires the file to be served over HTTP on the localhost and again spinning up a web server in the unit test seems like an overkill.

Is there any way to unit test this custom selector without all these dependencies?