It was in 1999 October or November; Prof. Mitra visited our Institute to deliver an invited lecture on his work. It was my privilege and honor to host his lecture. We had a brief discussion on his research work and what I was doing then. As I could recollect today, I asked him, which contribution he considered to be the most impactful among his many in areas of signal processing. He mentioned about his work on multi-rate digital signal processing. His talk was also primarily centered on this topic. I was working then on color processing, in particular on color interpolation algorithms. He told me, “We are also working on this problem. There is a grant from National Semiconductor. Why don’t you come to UCSB in this summer?” I readily agreed, as I was also looking forward to some opportunity to spend a few months at a reputed University abroad. I got my new passport just a year before, and was yet to use it for a foreign travel. Immediately after his return to USA, he sent me a follow-up mail on my upcoming visit to his lab. It was a simple, informal mail, addressing me ‘Jayanta’, and asking me to send a few details to him for my visit, and at the end, he simply signed ‘Sanjit-da’. That is how Prof. Sanjit K. Mitra, a name among the pioneers of Digital Signal Processing, became Sanjit-da to me, and I am very fortunate to enjoy his friendship, guidance and research collaboration since then.
In fact, his invitation was also the first to me in any form from a foreign land. For the first time, I would be traveling outside India. Naturally, I was quite excited, and nervous. My whole family too! Though they did not accompany me that time. My son was then about three and a half years old, and my wife is a doctor in a Govt. hospital. For her getting leaves itself would have been a Herculean task! Sanjit-da probably sensed my naivety and primitiveness as a would-be visitor to USA. He informed me that he himself would come to Los Angeles to receive me and drive me to Santa Barbara. It boosted my spirit and confidence, but I did not realize then what greatness is attached to such an act! My lesson from him started from that very moment, as I introspect those events today.
So, on a sunny day in the first week of May, 2000, I landed at the Los Angeles airport with a grand welcome from the US immigration Department. It took around two hours to complete. Finally, when I came out from the exit door, I found Sanjit-da patiently waiting for me standing there. But there was no expression of irritation and annoyance in his body language for such a long waiting, that too expecting me to come out at any moment from the terminal! He simply greeted me with his habitual smile and calmness, ‘Welcome! I see, it took so much time in clearing your immigration.’ He drove me to his home at Goleta, about a three hour long drive. There I met Nandita Boudi. Those of us, who know both of them, we know how much they complement each other, and how caring and meticulous she is in welcoming her visitors! After our dinner, Sanjit-da took me to the house where he had arranged my accommodation, and introduced me with the land lady, a remarkable person, that I found later during my stay for about two and a half months in her house. After keeping my luggage there, he took me to a Super Market! Remember, everything was very new to me! India and America or USA in the year 2000 were like at two opposite poles, which the present generation may not be able to comprehend. Hardly was there any supermarket even in a metro city! And I live in a village called IIT Kharagpur Campus! I had no idea how the business runs there; how to get my toothpaste and toilet paper! Sanjit-da was very much aware about my ignorance. So, he explained me the processes of getting the stuff for my survival during my stay. My learning started from picking a shopping trolley, then going through the Aisles of different categories of items in the stores, choosing the items for my breakfast, lunch and dinner including a few suggested frozen semi cooked boxes with the instructions for heating them, then going to the counter for payment, and finally returning the trolley at some designated places. The whole process he carefully explained and guided me to go through. I was also a keen student and follower to all these very vital information!
Next Morning, Sanjit-da picked me up from my new temporary home and brought me to his Lab in UCSB. He introduced me with his visitors and students, showed my desk and computer, and then took me to the office for getting copies of lab and entrance keys issued to me. Likewise he went on carefully filling every dot of requirements for my short stay and work there. He took me to a bank in Goleta to open my account, to the Social Security Administration office in the downtown Santa Barbara for my Social Security Number and Card, etc. etc.
I am not sure whether some of you are wondering what I would be talking here, as my title of talk is ‘My research work with Prof. Sanjit K Mitra’. Naturally, you expect to hear about the problem statements, theory, maths, logic, algorithms, results, solutions, contributions, etc. etc.; usual items of any such research work; that too bearing the stamp of Prof. Sanjit Mitra! I assure you they do exist! But what I want to impress upon you, that, that is not all! A research work with Prof. Mitra is something more. It is more holistic. It not only enlightens you on a technical subject matter, but also sets examples of a person who takes so much care for his students and visitors, that in spite of being a very busy and demanding top class academician and researcher in his area, he patiently waits for a young visitor from India for more than two hours at the exit gate of an airport terminal to receive him. This is how a true research collaboration develops with mutual friendship and respect. So, it continued and is still going on. I am grateful for his continued mentorship and guidance. After my first visit, I was there in summers of 2001, 2003, and 2004. In 2002 I visited the Technical University of Munich (TUM) as a Humboldt Research Fellow. Late Prof. Manfred Lang kindly agreed to host me on Sanjity-da’s request. Three of us collaborated during that period. In 2007, I spent more than four months in the University of Southern California (USC), Los Angeles, hosted by him. Next year (2008), I visited him again, and we attended ICIP in San Diego. In that year itself, I was a Co-Program Chair of Indian Conference on Vision Graphics and Image Processing (ICVGIP) held at Bhubaneswar. Sanjit-da kindly agreed to become its General Chair and helped us in many ways in arranging distinguished speakers and top researchers to attend the conference. We also had joint papers in 2009 ICIP at Cairo, Egypt. In 2017, Sanjit-da and Prof. Liviu Goras of the Technical University of Iasi, Romania organized a workshop at Bucharest, and the International Symposium on Signals, Circuits and Systems (ISSCS) at Iasi. I participated there with their kind invitations. Apart from these, I made various short trips to visit Sanjit-da and Boudi in Santa Barbara. In fact, whenever I had an opportunity to travel to USA, I always loved to meet them, and cherished many such moments over these two and half decades. The last one was in the June last year, when I went to meet them with my son. He himself came to the Santa Barbara airport to receive us, as well as drove us around, treated us in restaurants, and finally on our departing day saw us off at the Santa Barbara Air Bus Terminus, at Goleta. Boudi, in spite of her critical and painful brain surgery that she went through a few months before, came to greet us during our stay. Age is just a number to both of them.
Now let me talk something about our research work. In my first visit, we worked with the development of color interpolation algorithms to be run under the constraints of low memory usage[1]. This work I presented in a conference in Kuala Lumpur, Malaysia in 2001. We also worked with a color enhancement algorithm, using techniques of saturation and desaturation of color pixels. This was carried out in collaboration with one of his visitors from Italy, Luca Lucheese. The idea of saturating the color of a pixel by moving its normalized x-y chromaticity point toward an edge of the gamut triangle struck me around 1992, and I also found some results then. When I discussed this approach with Luca, he pointed out that maximally saturated color points not necessarily bear the desired perceptual quality of a color. Rather, it may be a good idea to consider a point in between the white chromaticity point and this maximally saturated point at the edge of the triangle. Thus, we considered both the saturation and desaturation operations for balancing the color contrast in an image. This work was presented by Luca in ICIP’2001 in Thessaloniki, Greece[2].
The following summer (2001), Sanjit-da suggested me to look into the development of computational techniques directly in the block DCT domain for processing images. Eventually it turned out to be the primary focus of our research collaboration over more than a decade. He gave me a paper to read on image resizing authored by Rakesh Dugad and Narendra Ahuja[3]. He also suggested to use results of one of his prior work on the properties and relationship of sub-band DCTs. This work he did with his students S H Jung and Debarghya Mukherjee[4], and showed its application for low bit rate coding and decoding of images. We applied the same for image doubling and halving in the block DCT domain. We published this work in IEEE Transactions on Circuits and System for Video Technology[5], and also presented the work in a conference[6] in Moscow in 2002, which I attended from Munich. This work was among a few early works in the area of compressed domain processing of images and videos. Later with further generalization of this concept, we developed algorithms for image resizing with arbitrary factors in the block DCT domain. We have shown how the resizing problem in the block DCT domain is systematically factored into three basic tasks, namely composition of adjacent blocks into a single large block, decomposition of a block into a set of non-overlapping smaller blocks, and sub-band approximation of DCT coefficients while down-sampling or up-sampling. For example, for down-sampling an image we may merge adjacent blocks into a large block and then apply the sub-band approximation to down sample it in a smaller size block. On the other hand, for up-sampling, we may first decompose a block into a number of small adjacent blocks, each of which are up-sampled using sub-band approximation. A combination of composition and decomposition by choosing suitable numbers of blocks to merge or to be fragmented into, provides us resizing of images by arbitrary ratios. As all these operations are linear, the whole computation is reduced to a single linear operation by using a composite matrix of transformation. In particular, in this computational approach, our fundamental contribution was on determining the precise scale factor in sub-band approximation by applying the theory of sub-band DCT, which was overlooked in many other works and mostly decided there empirically without any explanation. We published this work in the IEE Proceedings of Vision, Image and Signal Processing[7], and also presented in ICIP’2004 at Singapore[8]. Later I included more general forms of resizing computation in my book on ‘Image and Video Processing in the Compressed Domain’ published by the CRC press in 2011[9]. I also presented those results in the ISSCS at IASI in 2017[10].
During my visit to the USC, Los Angeles in 2007, we worked on a simple idea, which was put forward in a few earlier works by other researchers to enhance color images in the block DCT domain. By simply scaling the DC and AC coefficients of a DCT block, it is possible to vary the contrast of an image. However, we hypothesized, that for maintaining ‘naturalness’ of a color image it is better to preserve the local contrast as a ratio of standard deviation and mean of the local distribution. Our hypothesis follows from the Weber law of our visual perception of an image. At the same time, determining an appropriate scale factor and its adjustment with the local content in an image are needed. With these simple observations and intuitions, we developed an algorithm for enhancing color images in the block DCT domain by scaling their coefficients. It was published in the IEEE Transactions on Image Processing[11], and was presented in the ICIP’2008 at San Diego[12]. This work has a good number of citations, and is taken as a benchmark technique of comparisons in a good number of works reported in the literature.
We also investigated various other computational problems in the block DCT domain such as filtering images[13], color interpolation[14] [15], color constancy[16], etc. In all these cases, our systematic treatment of block composition-decomposition and sub-band approximation in the block DCT domain, was integrated in the computational pipeline. For filtering images in the block DCT domain, we have applied the convolution-multiplication properties of DCTs, and derived composite filtering matrices for transforming a block in the original image to that in a filtered image.
Another interesting work, that we did in 2003, also attracted attention from many researchers in this domain. It considers direct compression of a color filter array (CFA) instead of compressing a full color image, that needs to be obtained by interpolating the CFA. In our technique, we have shown how antialiasing filters on the components derived from the CFA improve the performance of the compression technique. Sanjit-da’s PhD student Chin Co was the primary contributor of this work. This work was published in the IEEE Trans. on Consumer Electronics[17], and cited many a times.
These are the highlights of our research work, that we worked on mostly over a period of 10 years from 2000 to 2009. But our interactions did not stop there. It has been still going on. On several occasions, I sent him drafts of my articles for his review, and he always came back with his constructive comments and criticism on them.
At the end, I would like to express my deep gratitude to Sanjit-da for guiding and mentoring me at various stages of my life and career. I quite understand that I did not match his expectation in many cases, and offer my sincere apology for my limitations! But, it is my great privilege to enjoy his company and friendship, as I mentioned before and wish him and Boudi remaining as evergreen as they are!
30/3/2025
(Written for an invited talk delivered in ‘A Workshop in Honor of Professor Sanjit K. Mitra on his 90th Birthday.’, April 10, 2025 (Thursday), ICASSP’2025, Hyderabad.)
[1] J. Mukherjee, M.S. Moore, and S.K. Mitra, Color Demosaicing with Constrained Buffering, Proceedings of the 2nd International Symposium on Image and Signal Processing and Analysis, Kuala Lumpur, Malaysia, 14-16 Aug., pp. 52-55, 2001.
[2] L. Lucchese, S.K. Mitra, and J. Mukherjee, A New Algorithm Based on Saturation and Desaturation in the xy Chromaticity Diagram for Enhancement and Re-Rendition of Color Images, Proc. of Int. Conf. on Image Processing (ICIP), Thessaloniki, Greece, pp. 1077-1080, 2001.
[3] R. Dugad and N. Ahuja, A fast scheme for image size change in the compressed domain, IEEE Trans. Circuits Syst. Video Technol., vol. 11, pp. 461-474, Apr 2001.
[4] S.-H. Jung, S. K. Mitra and D. Mukherjee, Subband DCT: Definition analysis and applications, IEEE Trans. Circuits Syst. Video Technol., vol. 6, pp. 273-286, June 1996.
[5] J. Mukherjee and S. K. Mitra, Image resizing in the compressed domain using subband DCT, IEEE Transactions on Circuits and Systems for Video Technology, vol. 12, no. 7, pp. 620-627, July 2002
[6] J. Mukhopadhyay, J. and S.K. Mitra, Resizing of DCT-Based Compressed Images, Proc. of the 4-th Int. Conf. on Digital Signal Processing and Applications (DSPA), Moscow, Russia, 386-389, 2002.
[7] J. Mukherjee, and S.K. Mitra, Arbitrary resizing of images in the DCT space, IEE Proc. Vision, Image & Signal Processing, vol. 152, no. 2, pp. 155-164, 2005.
[8] J. Mukhopadhyay, and S.K. Mitra, Resizing of images in the DCT space by arbitrary factors, Int. Conf. on Image Proc.(ICIP), Singapore, Oct., pp. 2801-2804, 2004.
[9] J. Mukhopadhyay, Image and Video Processing in the Compressed Domain, CRC Press, 2011.
[10] J. Mukhopadhyay, Image Resizing in the compressed domain, International Symposium on Signals, Circuits and Systems (ISSCS), Iasi, Romania, 2017
[11] J. Mukherjee and S. K. Mitra, Enhancement of Color Images by Scaling the DCT Coefficients, IEEE Transactions on Image Processing, vol. 17, no. 10, pp. 1783-1794, Oct. 2008
[12] J. Mukhopadhyay and S.K. Mitra, Color Enhancement in the Compressed Domain, Int. Conf. on Image Proc. (ICIP-2008), San Diego, USA, Oct. 12 -15, pp. 3144-3147, 2008.
[13] J. Mukherjee, and S.K. Mitra, Image Filtering in the Compressed Domain, Indian Conference on Computer Vision, Graphics and Image Processing, ICVGIP-2006, Dec. 14-16, Madurai, India, LNCS-4338, pp. 194-205.
[14] J. Mukherjee, M.K. Lang, and S.K. Mitra, Demosaicing of images obtained from single-chip imaging sensors in YUV color space, Pattern Recognition Letter, vol. 26, pp. 985-997, 2005
[15] J. Mukherjee, M.K. Lang, and S.K. Mitra, False Color Suppression in Demosaiced Color Images, Indian Conference on Computer Vision, Graphics and Image Processing, ICVGIP-2004, Kolkata, Dec., pp. 320-325, 2004.
[16] J. Mukhopadhyay and S.K. Mitra, Color Constancy in the Compressed Domain, Int. Conf. on Image Proc. (ICIP-2009), Cairo, Egypt, Nov. 7-11, pp. 705-708, 2009.
[17] Chin Chye Koh, J. Mukherjee and S. K. Mitra, New efficient methods of image compression in digital cameras with color filter array, IEEE Transactions on Consumer Electronics, vol. 49, no. 4, pp. 1448-1456, Nov. 2003
