PreventDefault does not work for on with delegation

PreventDefault does not work for on with delegation

I'm trying to write some JQuery so that whenever the user clicks on a link
(i.e. the a tag), there's an alert, and the link does not actually jump to
another place.
$(document).ready(function() {
$(document.body).on('click',"a",function(event){
event.preventDefault();
alert("Hello!");
});
});
The links in this file are not actually in the HTML code, but rather they
are inserted by some other JavaScript. The alert works, but the default is
not prevented. How can this be fixed?