The computer you studied is ok, and there are also requirements for computer major in the tax civil service exam. The tax bureau also recruits computer personnel, which has an information center.
Once you go in, it has little to do with your major. Everyone has to study tax law and accounting, but those with a professional foundation learn it faster.
I majored in chemical engineering at university, and passed the exam to enter the tax bureau. We were not limited to majors at that time.