In Moore v. Bridgewater Twp., 69 N.J. Super. 1, 173 A.2d 430 (App.Div.1961), after restating the basic premise that "while nonconforming uses may be continued as of right they are to be restricted, and may not be enlarged as of right, except where the enlargement is insubstantial or where a variance therefore has been granted," 69 N.J. Super. at 10-11, 173 A.2d 430, the Court distinguished the extraction of a diminishing asset from other uses of land: We must not lose sight of the fact that the instant case deals with the removal from land of a natural resource or a diminishing asset, as distinguished from a structure placed upon land or some use made by the owner upon his land. This opinion, of necessity, must be concerned and limited only to its own facts dealing with a diminishing asset. Id., at 11, 173 A.2d 430. It is quite obvious that an owner intending to carry on a quarrying operation acquires more land than he thinks he will need so that he will not be a source of nuisance to his neighbors. For practical and economical reasons he must begin operations at one given point and continue from there to a point on his lands where his natural resource ends or at his boundary line. For the same reasons, it is not feasible for him to quarry at different locations at the same time. 69 N.J. Super. at 15, 173 A.2d 430. Nevertheless, we recognized there must be some "outward manifestation of . . . intent" and not merely an "unexpressed intention or hope" on the part of the landowner that he intends to utilize an entire parcel for what has become a nonconforming use. Moore, 69 N.J. Super. at 16, 173 A.2d 430.