© 2023 Taylor & Francis Group, LLC.It has become crucial to develop new adsorbents to remove toxic dye wastes from wastewater to protect our existing aquatic ecosystem. Here, to remove methyl orange azo dye from aqueous solutions, a new p(HEMA-co-AETAC)/VMt nanocomposite hydrogel was prepared by using 2-hydroxyethyl methacrylate and [2-(acryloyloxy)ethyl]trimethylammonium chloride monomers using vinyl-based montmorillonite crosslinker. Obtained new types of crosslinker and nanocomposite hydrogels have been characterized by various analysis methods. In addition, the effects of crosslinker amount, monomer amount and pH on the swelling behavior of nanocomposite hydrogel were investigated. The effects of essential parameters such as contact time, initial dye concentration, pH effect of the solution, adsorbent amount, and temperature on adsorption were evaluated. It was determined that there was no significant change in the adsorption ability of the nanocomposite hydrogel in the pH range of 3–10. Experimental results showed that the Langmuir isotherm model fit better, and the calculated maximum adsorption capacity was 961.54 mg/g. In addition, it was determined that vinyl-based montmorillonite used as a crosslinker supported MO adsorption. p(HEMA-co-AETAC)/VMt nanocomposite hydrogel may be a new type of adsorbent to remove toxic synthetic dye residues from aqueous media.