my coding doesn't work plz help

[Ofnuts]

It is possible to define a function into another function but is pointless in your case. Rid of all these math functions which are doing one thing, to return a calculation result of two values. There is no indentation of elif to if

  if(operation == '+'):
      return num1 + num2
  elif(operation == '-'):
      return num1 - num2
  elif(operation == '*'):
      return num1 * num2
  else:
      return num1 / num2
......

print(calculator())

Also, you call calculator() inside its own code. Use while loop instead.
For now, that's all.

All in all, I'd have a different approach to almost everything.

On the contrary... Using functions is a good idea. You then use a dictionary where the keys are the operator strings and the value is the matching function:

ops={'+':add,'-':subtract,'*':multiply}

and use as:

result=ops[operator](num1,num2)

(python has built-in functions for the base operators (see the operators module), so you can even avoid defining some of the functions)