Bias Check for Smart Vision‑Language Models

Large vision‑language models are getting smarter, but they can still favor certain groups. Researchers noticed that the tools used to spot these biases were limited in size and scope. To fill that gap, a new test set called VLBiasBench was created. The benchmark covers nine common bias themes: age, disability, gender, nationality, looks, race, religion, job type, and wealth level. It also examines two mixed categories—race with gender and race with wealth—to see how overlapping identities affect results.The data set is built by letting a powerful image generator produce almost 47, 000 pictures that match each bias scenario. These images are paired with questions of two kinds: open‑ended and multiple choice, giving a broad look at how models respond. In total, the collection holds more than 128, 000 unique image‑question pairs. The team tested fifteen free models and two commercial ones with this new benchmark. Their findings reveal surprising patterns of bias that were not obvious before, showing how important thorough testing is. The full test kit and its instructions are posted online for anyone to use.