Sunday, 15 October 2017

Removing Watermark


I am currently preparing a talk on Balintore Castle and came across an original 1860 drawing on the RIBA (Royal Institute of British Architects) website. This is great, well apart from the water mark! :-)


As I have a background in image processing, I though it would be fun to try to find an algorithm to remove the watermark. You can judge for yourselves how well I have done at the end of this blog entry. That's one way to keep you reading. :-)

Here is the original watermarked image:

original watermarked image of castle



There are essentially 3 different areas in the image:
  1. untouched area outside of letters
  2. black line around letters
  3. area inside letters with reduced colour and contrast

As the image inside the letters is black and white and has been reduced in contrast, I wondered if this could give any clues for reconstruction? I looked at the hue (colour channel) and the saturation (colour intensity channel). There was little discernible information in the hue, but the saturation image below was a revelation and give insight into the solution.

colour saturation channel
N.B. letters have near 0 colour saturation i.e. no colour information

You can see that the saturation inside the letters is essentially zero (shown by the deep black colour). This means that parts of the region inside the letters are distinguished by low saturation, and therefore can be treated differently in the restoration.

I threshold the saturation image looking for low saturation regions. By choosing a suitable value (around 47), everything below this was essentially a letter, shown by the coloured regions below. I have coloured connected regions different colours and this information is useful in the next stage.

letters thresholded from saturation and split into connected regions
In the image above there is some noise i.e. small blobs that do not form part of a letter. By eliminating small regions (less than 170 pixels), then one arrives at the 7 large regions below. Letters successfully detected! :-)



small "noise" regions removed
By growing and shrinking the letter regions, by image processing techniques called dilation and erosion respectively, one can create an edge region as below. It is important that this edge region totally covers the black letter boundary  so we can remove this. By trial and error, I had to do 1 shrink operation and 2 grow operations.



edge mask created from large connected regions
The image below may look little different from the starting-point watermarked image. However, the black boundary is now the "edge mask" I have created. Inside the letters I have increased the contrast of the black and white image to exactly match the contrast of the surrounding sepia image. This is done by a "histograming technique" making the histogram of intensity distributions of the inner image match that of the outer image.
As well as the "edge mask", I have a "letter mask" for the inside of the letters which is essentially the same as the earlier coloured blobs.


image outside letters + edge mask + contrast enhanced image inside letters 
I now want to fill in the missing detail! :-) Inside the letters I need to supply the missing hue and saturation information. In the edge mask region I need to supply the missing colour, saturation and intensity information. Thankfully there is an image processing technique called "in painting" for filling in missing regions. You will be familiar with this technique, as it the one used to digitally removes scratches from photographs.

So in simple terms, the hue and saturation information is in-painted inside the letter masks, and the intensity information is in-painted inside the edge mask. The eye is less sensitive to hue and saturation information so the large degree of inaccurate guesswork inside the big letter regions is not a problem. However, when intensity information is in-painted inside the letter mask, this looks dreadful. The eye is incredibly sensitive to intensity information. It was therefore necessary to in-paint intensity in the smallest possible region  i.e. just the edge mask, and to use the intensity information inside the letters - which demanded the complex histogram work!


missing parts of image filled-in


Anyhow, how did I do? You can see the resulting final in-painted image above.  Some faint echos of the watermark are visible on the left hand side of the image. If I was a digital artist I could paint these out. :-) But as a mere programmer, the point of diminishing returns has been reached and this is the best I can do on a Sunday morning! :-) The code is appended.



import numpy as np
import os
import random
import cv2
from skimage.morphology import binary_dilation, binary_erosion

def hist_match(source, template, ignore_black = True):
    """
    https://stackoverflow.com/questions/32655686/histogram-matching-of-two-images-in-python-2-x
    Adjust the pixel values of a grayscale image such that its histogram
    matches that of a target image

    Arguments:
    -----------
        source: np.ndarray
            Image to transform; the histogram is computed over the flattened
            array
        template: np.ndarray
            Template image; can have different dimensions to source
    Returns:
    -----------
        matched: np.ndarray
            The transformed output image
    """

    oldshape = source.shape
    source = source.ravel()
    template = template.ravel()

    # get the set of unique pixel values and their corresponding indices and
    # counts
    s_values, bin_idx, s_counts = np.unique(source, return_inverse=True,return_counts=True)
    if ignore_black:
        s_counts[0] = 0

    t_values, t_counts = np.unique(template, return_counts=True)
    if ignore_black:
        t_counts[0] = 0

    # take the cumsum of the counts and normalize by the number of pixels to
    # get the empirical cumulative distribution functions for the source and
    # template images (maps pixel value --> quantile)
    s_quantiles = np.cumsum(s_counts).astype(np.float64)
    s_quantiles /= s_quantiles[-1]
    t_quantiles = np.cumsum(t_counts).astype(np.float64)
    t_quantiles /= t_quantiles[-1]

    # interpolate linearly to find the pixel values in the template image
    # that correspond most closely to the quantiles in the source image
    interp_t_values = np.interp(s_quantiles, t_quantiles, t_values)

    returned_image = interp_t_values[bin_idx].reshape(oldshape)
    return returned_image.astype(np.uint8)

def coloured_image_to_edge_mark(coloured_image):
   image_sum = coloured_image[:,:,0] + coloured_image[:,:,1] + coloured_image[:,:,2]
   mask = image_sum > 0
   return mask

def triple_mask(mask):
    return np.stack( [mask]* 3, axis = 2)

def get_inner_and_outer_masks(mask):
    inner_mask = binary_erosion(binary_erosion(binary_dilation(mask)))
    inner_pixel_count = np.count_nonzero(inner_mask)
    #inner_mask = mask
    outer_mask = binary_dilation(binary_dilation(mask)) # no colour abnormaility
    outer_pixel_count = np.count_nonzero(outer_mask)
    print("inner_pixel_coint = ",inner_pixel_count)
    print("outer_pixel_count = ",outer_pixel_count)
    return inner_mask, outer_mask

def balance_histograms_using_v(inner, outer):
    """
    make RGB image inner have the same brightness (i.e. v) histogram as image outer
    """
    inner_v_before, inner_hsv = rgb_to_intensity(inner)
    outer_v,        outer_hsv = rgb_to_intensity(outer)
    inner_v_after = hist_match(inner_v_before, outer_v)
    inner_hsv[:,:,2] = inner_v_after                   # edit V channel only
    return cv2.cvtColor(inner_hsv, cv2.COLOR_HSV2BGR)  # return as BGR

def fill_in(io, edge_mask, outer_mask):
    """
    http://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_photo/py_inpainting/py_inpainting.html
    """
    fill_in_method = cv2.INPAINT_TELEA # other choice cv2.INPAINT_NS - makes little visible difference
    io_hsv        = rgb_to_hsv(io)
    h_before      = io_hsv[:,:,0]
    s_before      = io_hsv[:,:,1]
    v_before      = io_hsv[:,:,2]

    outer_mask_uint    = np.where(outer_mask,255,0).astype(np.uint8)
    s_after   = cv2.inpaint(s_before, outer_mask_uint, 15, fill_in_method)       # use outer mask to fill in saturation
    h_after   = cv2.inpaint(h_before, outer_mask_uint, 15 ,fill_in_method)       # use outer mask to fill in hue
    v_after   = cv2.inpaint(v_before,       edge_mask,  2, fill_in_method)  # use edge to fill in hue

    io_hsv[:,:,0] = h_after
    io_hsv[:,:,1] = s_after
    io_hsv[:,:,2] = v_after
    return hsv_to_rgb(io_hsv)

def rgb_to_hsv(im):
    return cv2.cvtColor(im, cv2.COLOR_BGR2HSV)

def hsv_to_rgb(im):
    return cv2.cvtColor(im, cv2.COLOR_HSV2BGR)

def rgb_to_intensity(im):
     hsv  = rgb_to_hsv(im)
     return hsv[:,:,2], hsv

def make_random_colour_map_with_stats(stats, pop_thresh = 0):
    n = len(stats)
    colour_map = np.zeros( [n, 3], dtype=np.uint8)
    for i in range(n):
        if ( (pop_thresh != 0) and (stats[i][4] < pop_thresh) ) or  (i == 0):
             colour_map[i] = [0,0,0]                            # make small regions and region 0 (background) black
        else:
            for j in range(3):
                colour_map[i,j] = 1 + random.randint(0,254)     # big regions are a non-zero random colou
    return colour_map

"""

Image comes from here

https://www.architecture.com/image-library/RIBApix/licensed-image/poster/balintore-castle-angus-the-entrance-front/posterid/RIBA65186.html
"""

def display_and_output_image(name, im):
    cv2.imshow(name,im)
    file_name = os.path.join( "C:\\Users\\david\\Desktop\\", name + ".jpg")
    cv2.imwrite(file_name,im)


def create_letter_mask(image_saturation):
    """
    https://stackoverflow.com/questions/35854197/how-to-use-opencvs-connected-components-with-stats-in-python

    threshold saturation to detect letters (low saturation)
    find big connected components (small connected components are noise)
    """
    connectivity = 4
    ret, thresh_s = cv2.threshold(image_saturation, 42, 255, cv2.THRESH_BINARY_INV)  # 50 too high, 25 too low
    output = cv2.connectedComponentsWithStats(thresh_s, connectivity, cv2.CV_32S)
    blob_image = output[1]
    stats = output[2]
    pop_thresh = 170
    big_blob_colour_map = make_random_colour_map_with_stats(stats, pop_thresh)
    all_blob_colour_map = make_random_colour_map_with_stats(stats)
    big_blob_coloured_image = big_blob_colour_map[blob_image]                       # output
    all_blob_coloured_image = all_blob_colour_map[blob_image]                       # output
    display_and_output_image("big_blob_coloured_image", big_blob_coloured_image)
    display_and_output_image("all_blob_coloured_image", all_blob_coloured_image)
    letter_mask = coloured_image_to_edge_mark(big_blob_coloured_image)
    return letter_mask

def main():
    """
    original image comes from here

    https://www.architecture.com/image-library/RIBApix/licensed-image/poster/balintore-castle-angus-the-entrance-front/posterid/RIBA65186.html
    """
    im = cv2.imread(r"C:\Users\david\Desktop\riba_pix_cropped.jpg")
    print (im.shape)
    display_and_output_image("image",im)
    hsv = rgb_to_hsv(im)
    image_saturation = hsv[:,:,1]                                                           # output
    display_and_output_image("image_saturation",image_saturation)

    letter_mask = create_letter_mask(image_saturation)

    # outer mask bigger than letter mask
    # inner mask smaller than letter mask
    # edge mask is between inner and outer mask and contains black line round letters (i.e. to be removed)
    inner_mask, outer_mask =  get_inner_and_outer_masks(letter_mask)
    edge_mask = np.logical_and( np.logical_not(inner_mask), outer_mask)
    edge_mask = np.where(edge_mask,255,0).astype(np.uint8)
    display_and_output_image("edge_mask",edge_mask)

    inner_image = np.where( triple_mask(inner_mask), im, 0)
    outer_image = np.where( triple_mask(outer_mask) ,0 ,im)

    balanced_inner_image = balance_histograms_using_v(inner_image,outer_image)

    before_filling_in = balanced_inner_image + outer_image                                   # output
    display_and_output_image("before_filling_in",before_filling_in)

    after_filling_in = fill_in(before_filling_in, edge_mask, outer_mask)                     # output
    display_and_output_image("after_filling_in",after_filling_in)

    cv2.waitKey(0)
    cv2.destroyAllWindows()



if __name__ == '__main__':
    main()


Sunday, 1 October 2017

A Study In Pink

I am currently seeking listed building consent (LBC) to fix-up the interior walls in the main body of Balintore Castle. This is essentially all rooms not in the kitchen wing's ground floor, which is the only part of the building which has planning permission for full restoration.

Angus Council came back to me to say they needed plans with all affected rooms annotated. When I was around 4, the gift of a colouring-in book was as good as life got, and it is the tragedy of being a grown-up that one can see the pointlessness of colouring everything in when a short English phrase ("all rooms except the kitchen wing ground floor") does a better job.

Anyhow, please find appended my compliant pink colouring-in oevres . OK, I maybe enjoyed it just a little. :-)




annotated basement plan


annotated principal floor plan

annotated first floor plan
annotated attic plan


 

Monday, 18 September 2017

My Knobs' Patina


One of the eternal debates in restoration is "how much". We have all seen over-restored buildings which have lost their character. On the other hand, one would rather that one's restored building is distinctly distinguishable from a ruin. Where is the happy middle ground?

This, in microcosm, is the same debate when polishing brass. Too much polishing and it looks like new brass; too little polishing and the metal looks too tarnished.

Anyhow, I am currently polishing a job lot of brass door knobs. I have completed the 5 on the right stopping before I reach an overly polished look - which is often the flaw with modern brass items. The 4 on the left have some remaining lacquer coating, which accounts for the "gold" colour with brown regions where the  lacquer has worn off. In fact, I initially interpreted this look as gilding coming off and I avoided polishing to stop the loss of more gilding to expose the underlying unattractive surface, which I though might be iron.

The two in the middle have the full brown patina that brass develops over time: not unattractive but not the gleaming surface the Victorians aspired to. My personal preference is to have shiny highlights where human touch would naturally keep the surface shiny and tarnishing in the hollows. This combination celebrates the range of finishes and provides contrast.

In some people's eyes, I have over-polished the knobs. However, in time these will tarnish, so you can't really get it wrong. 

You can see some of the knobs have an appealing "rose brass" colour, where there is presumably a larger copper content.

And to end with a tip: wire wool is better and infinitely faster than Brasso to polish brass. :-)





polishing of my door knobs in progress

Wednesday, 13 September 2017

Victorian Hardcore

Yesterday, as we had access to a digger, we decided to do some landscaping of the grounds of Balintore Castle. When I say "landscaping" please put away all notions of Capability Brown: this was removing dead trees, old tree stumps and the scrubby overgrown on the bank above the tennis court that had sprung up over the last 50 years or so. Two bonfires were kept burning throughout the day to remove the organic waste.

To finish-up,  Gregor started to clear away the mud he had churned-up on the east castle drive. To his great delight he discovered the original Victorian hardcore track hidden beneath a century or more of accumulated earth and grass.

The Victorian track turned out to be considerably wider than the current overgrown path extending in the uphill direction for at least another yard, so Gregor excavated the path and cut away the muddy banks on each side for a full 100 yard stretch.

The result of yesterday's endeavors was to turn what has always been a narrow, dark and damp path into an impressive, light and airy drive. When Gregor comes back from his holidays and if we get access to the digger again, I would love the whole drive to be excavated in this way. Car access to the castle would be immeasurably improved.

Sadly, I have no before photos as the improvement was a spontaneous decision!

newly excavated east drive - looking east

newly excavated east drive - looking west





Tuesday, 12 September 2017

Underfloor Heating: The Pour

I have put a short video of the start of the screed pour at the end of this blog entry, so if you are the impatient sort, you can jump right there! 

To say I was having kittens this morning is an understatement: there were so many ways today's screeding of the kitchen wing floor could go wrong.

I had followed the company's guidelines on arranging the sub-floor construction for a minimum screed depth of 50mm. However, I discovered on the company's costing spreadsheet just last night that they were only going to supply 40 mm! Their area estimate of 111 m2 gave in consequence their volume estimate of 4.4 m3. The company were only bringing 4.75 m3, so their tolerances were already very tight.

In last night's panic I did my own calculations using multiple depth measurements to get an average screed depth for each room. The kitchen in particular introduces huge uncertainty as the floor height is all over the place because it is above basement vaulting. The sub-floors of all the other rooms in the kitchen wing were installed at different depths so nothing was standard. This was despite me emailing plans to try to ensure consistency! :-) Anyhow, my own calculations gave a figure of 4.5 m3, and I breathed a sigh of relief as I was in the company's window. However, I had a niggling doubt that the screeding would run out because of the previously mentioned discrepancy, and dashed off an email to the company about the inconsistent depths in their documentation. I hoped this would cover my back.

Gregor and Andrew arrived early to hold my hand, which was very kind of them. However, when the workmen arrived from the screeding company, Andrew and Gregor headed off to do some landscape gardening in the grounds of the castle. No-one else wanted to take responsibility for the screeding so I had to be on site myself. Indeed supervising the screeding was the principal reason for my current stay at the castle, and I had known for years that no-one else wanted to do it. This is as it should be of course – I was paying the money – but it didn't make it any less nerve-racking.

I showed the workmen the room layout, the finished level I wanted and explained the existing irregularities in levels e.g. stone slabs at opposite ends of rooms which you would assume were at the same level, were out by up to 13mm. I even started miming with a slate tile, assuming that gesture and show would remove any ambiguity in what I was saying. One of the workmen must have picked-up my nervousness, as he said “We do this every day, we know what we are doing.”. I had no doubt they knew what they were doing, but was concerned they were doing it with my money :-) and that I had not communicated properly the non-standard requirements of the castle's restoration.

Before any pouring, the workmen placed a large number of small metal tripods in each room. Using a laser level the workmen screwed down a circular plate built into the tripod, to the final floor level. The pour would be complete when it reached the level of these circular plates. Using the measurements obtained the workmen came up with a figure of 5 m3. I was warned that, in consequence, they might need to fetch another lorry-load of screed and that this would cost me an arm and a leg. My panic increased!

The workmen wanted me to tape a few bits of pipe and a few lengths of expansion strip down to ensure they would not float during the pour. I was eager to oblige as I just didn't want anything to go wrong. During the pour, they said that by going just 2 millimetres lower in each room, that they thought they could manage with 4.75 m3. I readily agreed! There was another panic in the former scullery, as they had to go higher than the requested floor level to ensure some pipes would be properly covered - but would this cause the screed to run out?

So yes, I was on hand to make some quick real-time decisions – again as it should but my adrenalin was definitely on high! My mouth was going dry, and when your body responds, you know you are giving birth to kittens. :-)

There was a hump in one part of the kitchen floor, isolating a low area which I had identified as problematic. One of the workmen found this and asked if I wanted him to kick some screed in there. My response: “Yes, please!”

At one stage, I could see some pipes in the kitchen rising slightly above the surface of the screed. This is called “crowning” and I was trying to assess how bad it would be. However, the workmen do a second pass where they spray the screed with a hardener and tamp the surface with a T-shaped tool. Thankfully the crowning disappeared after this.


The whole pour only took about 30 minutes – much quicker than the preparations and indeed much quicker than the subsequent cleaning up. As the pumping finished in the last room with the workmen running out of screed, it looked to a first approximation that there had just been enough screed and that the levels were about right. I deliberately did not look too closely as what had been done was now a fait-accompli and I could check things over accurately once the screed had dried. I was too worried about falling into the liquid screed, and spoiling the good judgement and care which had accompanied the pour.

I was asked if they could pump out the remaining contents of the wide bore pipe on the grass. I asked “Well how much is in there?” as I didn't want to pollute the castle grounds. It turned out there would be quite a lot, so I asked if they could fill some containers instead and I could use this to fill voids in the entrance hall floor. We had cleared out the entrance hall just in case there was any remaining screed. In the end we managed to get a couple of full barrow loads on the entrance hall floor. This was some last minute running around I was not expecting – and yes I was panicing! :-)

The most important thing, I guess, is that the membrane Andrew installed did not leak – hurrah! We had gone round this a number of times taping any holes or places where leaks might occur.

In fact the screeding is such a major and long anticipated landmark that I cannot believe it has actually happened. Acceptance may take some time, perhaps after 48 hours when one can actually walk on it? I will do another blog entry with the "after pour" pictures.

Huge thanks to Danny and Norbert of East-West Flooring for bringing their professional and good-humoured charm to a stressed-out castle laird!




kitchen tripod invasion 
scullery tripod invasion

pantry tripod invasion

bowser from front

bowser from rear

bowser from side

 Danny (bucket) and Norbert (hose) in action


Sunday, 10 September 2017

Underfloor Heating: Ready for Pour

The pour of the liquid screed floor in the kitchen wing at Balintore Castle is teetering on the brink of becoming a reality. :-) This has been delayed 3 years by a variety of factors, out of my control sadly, and this part of the castle has in consequence been frustratingly unusable for all of that time as everything had been pulled out in preparation. Things get real this Tuesday when the workmen arrive, so fingers crossed there are no leaks in the black plastic membrane, or the wine cellars beneath will be filled at £300 per extra cubic metre !

This blog post forms a room-by-room visual record of the kitchen wing before the pour and could form a useful resource when locating the pipes in the future. If there are two views, one will be the reverse angle to get all the pipework on camera.


Many thanks to Andrew for his sterling work in laying the pipes - he invented two special tools to get the job done! :-)

Covered Walkway

Note Andrew's wiggle when an odd number of runs fits the width! The boards protect the pipe as we need access through the utility room door.



Bedroom 1 (former Coal Cellar)


Utility Room (former Dairy Larder)


Bathroom (former Meat Larder)


Bedroom 2 (former Pantry)




Bedroom 3 / Snug (former Scullery)



Kitchen/Living Room (former Kitchen)


Under my intimations of its superior heat distribution, Andrew branched out from his pedestrian boustrophedon to a archimedean spiral, nay, a double spiral due to the size of the kitchen floor area.




Corridor Radiators

The slab floor is intact only in a single room in the kitchen wing. So for this room (an interior corridor) we had to use two large radiators instead of underfloor pipes. These radiators and a few others were obtained for the princely sum of £1.19 on eBay and fit the window alcoves perfectly. This is not a coincidence, this was years of searching! :-) The radiators come from a mansion flat next to the Albert Hall in London. Needless-to-say the shipping was the more expensive part of the bargain.