Improving Number Detector

My current goal is doing what this Keras.js demo does with Tensorflow.js. Looking at their GitHub repossitory I noticed that it uses Vue.js meaning that was complicated to convert to plain Javascript but that also gave me an idea, I’ll convert the previous code to TensorFlow.js! (and chaatbots helped)

It’s still getting numbers wrong 😅️ but it works.

Here’s the impotant part:

  // Get the predictions for the canvas data.
  const imgData = ctx.getImageData(0, 0, CANVAS_SIZE, CANVAS_SIZE);
  const input = new onnx.Tensor(new Float32Array(imgData.data), "float32");

  const outputMap = await sess.run([input]);
  const outputTensor = outputMap.values().next().value;

  const predictions = outputTensor.data;
  const maxPrediction = Math.max(...predictions);

became

const imgData = ctx.getImageData(0, 0, CANVAS_SIZE, CANVAS_SIZE);
const imageTensor = tf.browser.fromPixels(imgData);
const resizedImage = tf.image.resizeBilinear(imageTensor, [28, 28], true);

// Convert the resized image to grayscale
const grayscaleImage = tf.image.rgbToGrayscale(resizedImage);

// Expand dimensions for the model input
const batchedImage = grayscaleImage.expandDims(0);
const normalizedImage = tf.cast(batchedImage, 'float32').div(tf.scalar(255));

const predictions = await model.predict(normalizedImage).dataSync();
const maxPrediction = Math.max(...predictions);

Nicely done but the detection, while better, wasn’t as good as I expected. Just try it.