Can someone help me solve this ?

dF/dx = d/dx[f(xy^2 + x^2)] = f'(xy^2 + x^2)*d(xy^2 + x^2/dx = f'(xy^2 + x^2) * (y^2 + 2x)

Not knowing anything about f this is the best we can do.

We can write this using a substitution by letting w = xy^2 + x^2:

so now F(x,y) = f(w) and by the chain rule dF/dx =d[ f(w)]/dx = f'(w) * dw/dx

which is more compact way of writing it.

It might be easier to see how this works if we use a specific example for f(w), say some thing easy like f(w) =2w^2 where w = xy^2 + x^2

Now we have 2 choices to proceed and will get the same answer both ways:

1) dF/dx = f'(w) * dw/dx

= [4w]*[y^2 + 2x] --- holding y constant

= [4xy^2 + 4x^2]*[y^2 + 2x]

= 4xy^4 + 8x^2y^2 + 4x^2y^2 + 8x^3

= 4xy^4 + 12x^2y^2 + 8x^3

2) Find F in terms of x and y first and then differentiate:

f(w) = 2w^2 where w = xy^2 + x^2

F(x,y)= f(w)

= 2w^2

= 2[xy^2 + x^2]^2

= 2[x^2y^4 + 2x^3y^2 + x^4]

= 2x^2y^4 + 4x^3y^2 + 2x^4

Keeping y constant and differentiating wrt x:

dF/dx = 4xy^4 + 12x^2y^2 + 8x^3

Same for dF/dy