All You Need To Implement Search By Image

Thushara Wedagedara

3 years ago

Microsoft Azure is a cloud platform with more than 200 products to find a solution to your problem. Among those, Cognitive Services are some amazing products that provide you with many AI functionalities. For example:

Anomaly detector
Content moderator
Personalizer
Language understanding
QnA maker
Text analytics
Translator
Speech service
Computer vision
Custom vision
Face API
Cognitive services multi-service account

With these resources, any developer can incorporate AI into their applications without the need for deep learning of machine learning. All of these can be accessed via the REST API and the Windows IDE, allowing them to be used in any modern application.

Table of Contents

What is the Bing Visual Search API?

With the Bing Visual Search API you can integrate image search functionalities into your application. Users will be able to search by supplying an image and the Bing search API will find images which are visually or thematically similar. Also, it can identify celebrities, places, items, and other related objects to the source image. You can even extract barcodes and text from images. It’s a pretty powerful resource.

How can I get the subscription key to use the Bing Visual Search API?

Goto Azure Portal and sign-in. In the portal, type “Bing” in the search box and select “Bing Search v7” from the marketplace. Now enter a name, subscription, and a resource group and create a new resource. Then go to the newly created resource and under the “Keys and Endpoint” tab, you will find two subscription keys. Use that key in the next step.

How do I implement The Bing Visual Search API in Delphi?

We can easily implement Visual Search API through the Delphi REST components. We can set up the API at design time using these components:

TRESTClient
TRESTRequest
TRESTResponse

Do the following..

Create a new VCL application and drop a TRESTClient component and set the content type as “application/json”. Then drop a “TRESTRequest” component and set the method to “rmPOST”. Also, drop a “TRESTResponse” component to get the response. Also, you need a TIdHTTP component to download images in the response.

Our API URL is:

https://api.bing.microsoft.com/v7.0/images/visualsearch/

Also, you need an image file to post with the request. So drop a TOpenDialog component to open image files. Add a button to browse for images.

Now add some code for the OnClick event

procedure TfrmMain.btnBrowseClick(Sender: TObject);
var
 Picture: TPicture;
begin
 if not dlgOpenImage.Execute then
  exit;
 strImgPath := dlgOpenImage.FileName;
 btnSearch.Enabled := strImgPath &lt;&gt; '';
 Picture := TPicture.Create;
 Picture.LoadFromFile(strImgPath);
 imgSource.Picture := Picture;
 edImgPath.Text := strImgPath;
end;

procedure TfrmMain.btnBrowseClick(Sender: TObject);

var

Picture: TPicture;

begin

if not dlgOpenImage.Execute then

exit;

strImgPath := dlgOpenImage.FileName;

btnSearch.Enabled := strImgPath <> '';

Picture := TPicture.Create;

Picture.LoadFromFile(strImgPath);

imgSource.Picture := Picture;

edImgPath.Text := strImgPath;

end;

In the search button, first, we add the “Ocp-Apim-Subscription-Key” as a header parameter. Then we add the image to process as a pkFile parameter of the RESTRequest. Then we execute the request and parse the response. The response is in JSON format. We can use “TJSONObject” and “TJSONArray” objects to easily parse the response. Then we can download similar images and display them in our Delphi application.

The full search button Delphi code to search by image should look like this

procedure TfrmMain.btnSearchClick(Sender: TObject);
var
  lparam : Trestrequestparameter;
  imgProcessed: bool;
  jsonObj: TJSONObject;
  jsonTags, jsonActions, jsonValue: TJSONArray;
  MS: TMemoryStream;
  Picture: TPicture;
  I, x, y: integer;
  image: TImage;
begin
  for i:=0 to ComponentCount-1 do
    if (Components[i] is TImage) then
      if (Components[i] as TImage).Parent=scrlbxImages then
        freeandnil(Components[i]);

  memResponse.Lines.Clear;
  RESTClient.BaseURL := edAPIURL.Text;
  RESTRequest.Method:=rmpost;
  imgProcessed := false;
  try
    RESTRequest.Params.Clear;
    RESTResponse.RootElement := '';
    lparam := RESTRequest.Params.AddItem;
    lparam.name := 'Ocp-Apim-Subscription-Key';
    lparam.Value := edSubKey.Text;
    lparam.ContentType := ctNone;
    lparam.Kind := pkHTTPHEADER;
    //This one is Important otherwise the '==' will get url encoded
    lparam.Options := [poDoNotEncode];

    lparam := RESTRequest.Params.AddItem;
    lparam.name := 'image';
    lparam.Value := strImgPath;
    lparam.ContentType := ctIMAGE_JPEG;
    lparam.Kind := pkFile;
    lparam.Options := [poDoNotEncode];
    RESTRequest.Execute;

    if not RESTResponse.Status.Success then
      showmessage(RESTResponse.StatusText + ' ' +
        inttostr(RESTResponse.StatusCode))
    else
    begin
      memResponse.Lines.Add(RESTResponse.JSONText);
      jsonObj := RESTResponse.JSONValue as TJSONObject;
      jsonTags := jsonObj.Values['tags'] as TJSONArray;
      for I := 0 to jsonTags.Count - 1 do
      begin
        jsonObj := jsonTags.Items[I] as TJSONObject;
        jsonActions := jsonObj.Values['actions'] as TJSONArray;
        for x := 0 to jsonActions.Count - 1 do
        begin
          jsonObj := jsonActions.Items[x] as TJSONObject;
          if jsonObj.Values['actionType'].Value = 'PagesIncluding' then
          begin
            jsonValue := (jsonObj.Values['data'] as TJSONObject).Values['value'] as TJSONArray;
            for y := 0 to jsonValue.Count - 1 do
            begin
              if y&gt; 10 then
                Break;
              jsonObj := jsonValue.Items[y] as TJSONObject;

              MS := TMemoryStream.Create;
              http.Get(StringReplace(jsonObj.Values['thumbnailUrl'].Value, 'https', 'http',[rfReplaceAll, rfIgnoreCase]), MS);
              MS.Position := 0;
              Picture := TPicture.Create;
              Picture.LoadFromStream(MS);
              image := TImage.Create(self);
              image.Parent := scrlbxImages;
              image.Align := alTop;
              image.Height := 180;
              image.Picture := Picture;
              image.Center := True;
              image.Proportional := true;
              MS.Free;
            end;
          end;
        end;
      end;
    end;

  finally
  end;
end;

procedure TfrmMain.btnSearchClick(Sender: TObject);

var

lparam : Trestrequestparameter;

imgProcessed: bool;

jsonObj: TJSONObject;

jsonTags, jsonActions, jsonValue: TJSONArray;

MS: TMemoryStream;

Picture: TPicture;

I, x, y: integer;

image: TImage;

begin

for i:=0 to ComponentCount-1 do

if (Components[i] is TImage) then

if (Components[i] as TImage).Parent=scrlbxImages then

freeandnil(Components[i]);

memResponse.Lines.Clear;

RESTClient.BaseURL := edAPIURL.Text;

RESTRequest.Method:=rmpost;

imgProcessed := false;

try

RESTRequest.Params.Clear;

RESTResponse.RootElement := '';

lparam := RESTRequest.Params.AddItem;

lparam.name := 'Ocp-Apim-Subscription-Key';

lparam.Value := edSubKey.Text;

lparam.ContentType := ctNone;

lparam.Kind := pkHTTPHEADER;

//This one is Important otherwise the '==' will get url encoded

lparam.Options := [poDoNotEncode];

lparam := RESTRequest.Params.AddItem;

lparam.name := 'image';

lparam.Value := strImgPath;

lparam.ContentType := ctIMAGE_JPEG;

lparam.Kind := pkFile;

lparam.Options := [poDoNotEncode];

RESTRequest.Execute;

if not RESTResponse.Status.Success then

showmessage(RESTResponse.StatusText + ' ' +

inttostr(RESTResponse.StatusCode))

else

begin

memResponse.Lines.Add(RESTResponse.JSONText);

jsonObj := RESTResponse.JSONValue as TJSONObject;

jsonTags := jsonObj.Values['tags'] as TJSONArray;

for I := 0 to jsonTags.Count - 1 do

begin

jsonObj := jsonTags.Items[I] as TJSONObject;

jsonActions := jsonObj.Values['actions'] as TJSONArray;

for x := 0 to jsonActions.Count - 1 do

begin

jsonObj := jsonActions.Items[x] as TJSONObject;

if jsonObj.Values['actionType'].Value = 'PagesIncluding' then

begin

jsonValue := (jsonObj.Values['data'] as TJSONObject).Values['value'] as TJSONArray;

for y := 0 to jsonValue.Count - 1 do

begin

if y> 10 then

Break;

jsonObj := jsonValue.Items[y] as TJSONObject;

MS := TMemoryStream.Create;

http.Get(StringReplace(jsonObj.Values['thumbnailUrl'].Value, 'https', 'http',[rfReplaceAll, rfIgnoreCase]), MS);

MS.Position := 0;

Picture := TPicture.Create;

Picture.LoadFromStream(MS);

image := TImage.Create(self);

image.Parent := scrlbxImages;

image.Align := alTop;

image.Height := 180;

image.Picture := Picture;

image.Center := True;

image.Proportional := true;

MS.Free;

end;

finally

end;

Here is a screenshot of our finished demo Visual Search Delphi application

You can download the demo application from this link: https://github.com/checkdigits/SearchByImage_example