Cannot run Jest tests

I'm trying to learn Jest to do which I'm trying to test some stuff in my React application. It doesn't matter what tests they are, whether it's simple snapshots, or some test for authentication, or a 1+1=2 kind of test. Everything fails.

After running npm run test all I get is this:

● Test suite failed to run

    Cannot find module 'jest-matcher-utils' from 'to-be-in-the-dom.js'

    However, Jest was able to find:
        './to-be-in-the-dom.js'

    You might want to include a file extension in your import, or update your 'moduleFileExtensions', which is currently ['web.js', 'js', 'web.ts', 'ts', 'web.tsx', 'tsx', 'json', 'web.jsx', 'jsx', 'node'].

    See https://jestjs.io/docs/en/configuration#modulefileextensions-array-string

    However, Jest was able to find:
        './index.js'

    You might want to include a file extension in your import, or update your 'moduleFileExtensions', which is currently ['web.js', 'js', 'web.ts', 'ts', 'web.tsx', 'tsx', 'json', 'web.jsx', 'jsx', 'node'].

    See https://jestjs.io/docs/en/configuration#modulefileextensions-array-string

    However, Jest was able to find:
        'dist/extend-expect.js'

    You might want to include a file extension in your import, or update your 'moduleFileExtensions', which is currently ['web.js', 'js', 'web.ts', 'ts', 'web.tsx', 'tsx', 'json', 'web.jsx', 'jsx', 'node'].

    See https://jestjs.io/docs/en/configuration#modulefileextensions-array-string

      at Resolver.resolveModule (node_modules/jest-resolve/build/index.js:259:17)
      at Object.<anonymous> (../../node_modules/@testing-library/jest-dom/dist/to-be-in-the-dom.js:8:25)
    ```

This is my package.json:

{
  "name": "test",
  "version": "0.1.0",
  "private": true,
  "dependencies": {
    "bootstrap": "^4.4.1",
    "firebase": "^7.14.6",
    "jest-matcher-utils": "^26.0.1",
    "leaflet-geosearch": "^2.7.0",
    "mdbreact": "^4.26.0",
    "react": "^16.13.1",
    "react-bootstrap": "^1.0.1",
    "react-bootstrap-range-slider": "^0.3.2",
    "react-bootstrap-star-rating": "^3.5.5-alpha.0.3",
    "react-burger-menu": "^2.6.13",
    "react-dom": "^16.13.1",
    "react-final-form": "^6.4.0",
    "react-hook-form": "^5.6.1",
    "react-icons": "^3.10.0",
    "react-router-dom": "^5.1.2",
    "react-scripts": "3.4.1",
    "weather-icons": "git+https://github.com/erikflowers/weather-icons.git"
  },
  "scripts": {
    "start": "react-scripts start",
    "build": "react-scripts build",
    "test": "react-scripts test",
    "eject": "react-scripts eject"
  },
  "eslintConfig": {
    "extends": "react-app"
  },
  "browserslist": {
    "production": [
      ">0.2%",
      "not dead",
      "not op_mini all"
    ],
    "development": [
      "last 1 chrome version",
      "last 1 firefox version",
      "last 1 safari version"
    ]
  },
  "devDependencies": {
    "enzyme": "^3.11.0",
    "react-addons-test-utils": "^15.6.2",
    "react-test-renderer": "^16.13.1",
    "babel-jest": "^25.5.1"
  },
  "description": "This project was bootstrappd with [Create React App](https://github.com/facebook/create-react-app).",
  "main": "index.js",
  "author": "",
  "license": "ISC"
}

I've tried to edit this: "test": "react-scripts test" to: "test": "jest"

This allows some snapshots to get tested, but 18 tests fails with:

● Test suite failed to run

    Jest encountered an unexpected token

    This usually means that you are trying to import a file which Jest cannot parse, e.g. it's not plain JavaScript.

    By default, if Jest sees a Babel config, it will use that to transform your files, ignoring "node_modules".

    Here's what you can do:
     • To have some of your "node_modules" files transformed, you can specify a custom "transformIgnorePatterns" in your config.
     • If you need a custom transformation specify a "transform" option in your config.
     • If you simply want to mock your non-JS modules (e.g. binary assets) you can stub them out with the "moduleNameMapper" config option.

    You'll find more details and examples of these config options in the docs:
    https://jestjs.io/docs/en/configuration.html

    Details:

    C:\Users\Kalle\Desktop\Skola\1dv430\atriptomemorylane\src\components\__test__\firebase.auth.test.js:1
    ({"Object.<anonymous>":function(module,exports,require,__dirname,__filename,global,jest){import { signInWithEmailAndPassword, signOutFirebase, isAuthenticated } from "../../config";
                                                                                                    ^

    SyntaxError: Unexpected token {

      at ScriptTransformer._transformAndBuildScript (node_modules/@jest/transform/build/ScriptTransformer.js:537:17)
      at ScriptTransformer.transform (node_modules/@jest/transform/build/ScriptTransformer.js:579:25)

For good measure, I'll toss one test in, with some redacted data:

test("signInWithEmailAndPassword should throw error with wrong credential", async () => {
  let error = "";
  try {
    await firebase
      .auth()
      .signInWithEmailAndPassword("xxx@gmail.com", "xxx");
  } catch (err) {
    error = err.toString();
  }

  expect(error).toEqual(
    "Error: The password is invalid or the user does not have a password."
  );
});

Hope someone can point out what I'm doing wrong. This is driving me nuts!