How to execute my Add function in javascript

It seems you want x to be an object. The correct syntax for that is

var x = {
  add: function(a,b){return a + b},
  mul: function(a,b){return a * b},
};

Or if you insist on using an IIFE (because you're doing more things in that scope than you've shown), then you'd do

var x = (function () {
  function add(a,b){return a + b}
  function mul(a,b){return a * b}

  return {add, mul};
}());

Functions defined within other functions aren't automatically available to outside code. Their default is to exist the same duration as any other local variable, available for garbage collection when the outer function is done executing.

To keep them around, you'll need to specify how you want to make them available. One option for that is to return them, adding a structure to hold them both:

var x = function () {
  return {
    add: function(a,b){return a + b},
    mul: function(a,b){return a * b},
  };
}();

Though, so far at least, the outer function isn't strictly necessary and could possibly be removed:

var x = {
  add: function(a,b){return a + b},
  mul: function(a,b){return a * b},
};

See the comments inline below:

// You can set the function up as a "constructor function",
// meaning that an object instance will be constructed from the
// invocation of the function. Standard convention is to use
// PascalCase for constructor function names:
function X() {
  // Later, when an instance of this object is made,
  // members of the object can be gotten by identifying
  // them as going along with "this" object instance.
  // There is more to know about this (ie. properties are
  // created as I'm showing here, but methods are often 
  // added to the function's prototoype).
  this.add = function(a,b){return a + b};
  this.mul = function(a,b){return a * b};
}

// Use the function to construct an object instance
let instance = new X();

// Then, use that instance and its associated members.
// FYI: Don't use .innerHTML when you aren't getting/setting
// any HTML. Use .textContent for raw text.
document.getElementById("demo").textContent = "Add: " + instance.add(3,4);
document.getElementById("demo").textContent += " | Multiply: " + instance.mul(3,4);

<p>After a function has been stored in a variable,
the variable can be used as a function:</p>

<p id="demo">hello</p>

What you are doing here is defining a function named x, and defining two variables inside that function, named add and mul. Due to the rules that JavaScript follows when it comes to item scope (Especially in the context of functions, see the section Function Scope), these variables are inaccessible outside of the function. To access them, you will need to have the function return an object defining them:

var x = function() {
  var add = function(a,b){return a + b};
  var mul = function(a,b){return a * b};

  return {
    add: add,
    mul: mul
  };
}

These can now be accessed by calling the function, which will return these two values:

var y = x();
document.getElementById("demo").innerHTML = y.add(3,4);

In side function if you want to declare public variables to constuctor use this. And make sure to put new to create new instance of your constructor

var x = new function () {

  this.add = function(a,b){return a + b};
  this.mul = function(a,b){return a * b};

};
console.log(x.add(1,3));
console.log(x.mul(5,3));