How to train transfer-learning model on custom dataset? ValueError: Shape must be rank 4

0

Issue

I am trying to build a transfer learning model to classify images. The images are a gray-scale (2D). previously I used image_dataset_from_directory method to read the images and there was no problem. However, I am trying to use a custom read function to have more control and access on the data such as knowing how many images in each class. When using this custom read function, I get an error (down below) while trying to train the model. I am not sure about what caused this error.

part1: reading the dataset

import numpy as np
import os
import tensorflow as tf
import cv2
from tensorflow import keras

# neural network
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers.experimental import preprocessing

IMG_WIDTH=160
IMG_HEIGHT=160

DATA_PATH = r"C:\Users\user\Documents\chest_xray"
TRAIN_DIR = os.path.join(DATA_PATH, 'train')

def create_dataset(img_folder):
   
    img_data_array=[]
    class_name=[]
   
    for dir1 in os.listdir(img_folder):
        for file in os.listdir(os.path.join(img_folder, dir1)):
       
            image_path= os.path.join(img_folder, dir1,  file)
            image= cv2.imread( image_path, 0)
            image=cv2.resize(image, (IMG_HEIGHT, IMG_WIDTH),interpolation = cv2.INTER_AREA)
            image=np.array(image)
            image = image.astype('float32')
            image /= 255 
            img_data_array.append(image)
            class_name.append(dir1)
    return img_data_array, class_name
# extract the image array and class name
img_data, class_name =create_dataset(TRAIN_DIR)
target_dict={k: v for v, k in enumerate(np.unique(class_name))}
target_dict


target_val=  [target_dict[class_name[i]] for i in range(len(class_name))]

this part will produce A list that has a size of 5232. inside the list there are numpy arrays of size 160X160 (float 32)

part 2: creating the model

def  build_model():

    inputs = tf.keras.Input(shape=(160, 160, 3))
    
    x = Sequential(
    [
        preprocessing.RandomRotation(factor=0.15),
        preprocessing.RandomTranslation(height_factor=0.1, width_factor=0.1),
        preprocessing.RandomFlip(),
        preprocessing.RandomContrast(factor=0.1),
    ],
    name="img_augmentation",
    )(inputs)
    # x = img_augmentation(inputs)

    model=tf.keras.applications.EfficientNetB7(include_top=False, 
                                               drop_connect_rate=0.4,
                                               weights='imagenet',
                                               input_tensor=x)
    # Freeze the pretrained weights
    model.trainable = False
    
    # Rebuild top
    x = tf.keras.layers.GlobalAveragePooling2D(name="avg_pool")(model.output)
    x = tf.keras.layers.BatchNormalization()(x)
    
    top_dropout_rate = 0.2
    x = tf.keras.layers.Dropout(top_dropout_rate, name="top_dropout")(x)
    outputs = tf.keras.layers.Dense(1, name="pred")(x)
    
    # Compile
    model = tf.keras.Model(inputs, outputs, name="EfficientNet")
    optimizer = tf.keras.optimizers.Adam(learning_rate=1e-2)
    model.compile(
        optimizer=optimizer, 
        loss=tf.keras.losses.BinaryCrossentropy(from_logits=True), 
        metrics=["accuracy"]
    )
    
    return model


model = build_model()

part 3: train the model

history = model.fit(x=np.array(img_data), y=np.array(target_val), epochs=5)

the error I get:

 ValueError: Shape must be rank 4 but is rank 3 for '{{node 
EfficientNet/img_augmentation/random_rotation_1/transform/ImageProjectiveTransformV3}} = 
ImageProjectiveTransformV3[dtype=DT_FLOAT, fill_mode="REFLECT", interpolation="BILINEAR"]
(IteratorGetNext, EfficientNet/img_augmentation/random_rotation_1/rotation_matrix/concat, 
EfficientNet/img_augmentation/random_rotation_1/transform/strided_slice, 
EfficientNet/img_augmentation/random_rotation_1/transform/fill_value)' with input shapes: 
[?,160,160], [?,8], [2], [].

Solution

The problem in the code is that OpenCV reads the image in grayscale format, but the grayscale format of the image returned is not (160,160,1) but (160,160).

Because of this fact, the error is thrown.

I managed to replicate your problem by testing it locally.

Say we randomly train on 12 samples.

Possible input formats:

 #This one works
 1. history = model.fit(x=np.random.rand(12,160,160,3), y=np.array([1,1,1,1,1,1,0,0,0,0,0,0]), epochs=5,verbose=1) WORKS
 #This one works
 2. history = model.fit(x=np.random.rand(12,160,160,1),  y=np.array([1,1,1,1,1,1,0,0,0,0,0,0]), epochs=5,verbose=1) WORKS
 #This one fails
 3. history = model.fit(x=np.random.rand(12,160,160), y=np.array([1,1,1,1,1,1,0,0,0,0,0,0]), epochs=5,verbose=1) FAILS

(1) and (2) work.

(3) fails, yielding:

ValueError: Shape must be rank 4 but is rank 3 for ‘{{node
EfficientNet/img_augmentation/random_rotation_4/transform/ImageProjectiveTransformV2}} = ImageProjectiveTransformV2[dtype=DT_FLOAT, fill_mode="REFLECT", interpolation="BILINEAR"](IteratorGetNext,
EfficientNet/img_augmentation/random_rotation_4/rotation_matrix/concat,
EfficientNet/img_augmentation/random_rotation_4/transform/strided_slice)’
with input shapes: [?,160,160], [?,8], [2].

Therefore, ensure that your data format is in the shape (160,160,1) or (160,160,3).

As an alternative, after you you read the image with OpenCV, you can use

image = np.expand_dims(image,axis=-1)

to programatically insert the last axis (the grayscale).

Answered By – Timbus Calin

This Answer collected from stackoverflow, is licensed under cc by-sa 2.5 , cc by-sa 3.0 and cc by-sa 4.0

Leave A Reply

Your email address will not be published.

This website uses cookies to improve your experience. We'll assume you're ok with this, but you can opt-out if you wish. Accept Read More