Monday, August 20, 2018

Using PLUpload with ASP.Net


It's a fairly common use case scenario in which one must allow their users to upload files into a web system for various means. Of course, in the situations I find myself, I need to allow this to happen, and I don't necessarily have a maximum size allowed for the file to be processed.  It's been a while since I last looked into uploading files, and even then I just used the traditional Input Type='File' control that's a part of the standard suite of HTML4 form elements.

Of course I had to push up the number of bytes that were allowed up at any given time, but it was still the simple file upload control. 

Fast forward a few years, and I knew there had to be a better solution. Or at least I hoped there was. 

Prayer was involved as well.  Just saying. 

Anyways, I stumbled into a nice deep hole which involved things like chunking, iFrames, Silverlight and Flash movies.

With a day of research and writing JavaScript already behind me, I  decided that running face first into the brick wall of the exterior of the office would be more productive than continuing the path I was on.  

So, I started looking for controls. 

And quickly found the Telerik suite of controls. Of which I wanted just the one. Sadly, they don't sell them that way. To use the Telerik upload control, I would have to buy the other dozen or so controls I did not need, want or would ultimately use.  And the DevExpress version was bundled in the exact same way. 

So, I continued my search. 

And found PLUpload (https://www.plupload.com/).

A quick download, 10 minutes of reading documentation, and then about two hours of coding, I was streaming chunks of data up to my development web server, and into the database. 5 more minutes, I was then generating a link which I could click that would stream those chunks back to me as the image they started life out as. 

My .ASHX had a fairly straightforward logic built into it, and looked like this: 
 Sub ProcessRequest(ByVal context As HttpContext) Implements IHttpHandler.ProcessRequest
        context = context
        Request = context.Request
        Response = context.Response
        sessionId = Request.Cookies("uploadSessionId").Value
        If Request.Files.Count > 0 Then
            Dim fileUpload As HttpPostedFile = Request.Files(0)
            Dim fileName As String = fileUpload.FileName
            If String.IsNullOrWhiteSpace(fileName) Or String.Equals(fileName, "blob") Then
                fileName = If(Request("name"), String.Empty)
            End If 'String.IsNullOrWhiteSpace(fileName) Or String.Equals(fileName, "blob")
            Dim tstr As String = If(Request("chunks"), String.Empty)
            Dim chunks As Integer = -1
            If Not Integer.TryParse(tstr, chunks) Then chunks = -1
            tstr = If(Request("chunk"), String.Empty)
            Dim chunk As Integer = -1
            If Not Integer.TryParse(tstr, chunk) Then chunk = -1
            ' If there Then are no chunks sent the file Is sent As one this likely a plain HTML 4 upload (ie. 1 single file)
            If chunks = -1 Then
                fileName = IO.Path.GetFileName(fileName)
                If Not onUploadStart(0, 1, fileName) Then Return
                If onUploadChunk(fileUpload.InputStream, 0, 1, fileName) Then
                    onUploadComplete(fileName)
                    WriteSucessResponse()
                End If ' onUploadChunk(fileUpload.InputStream, chunk, chunks, fileName)
                Return
            Else ' chunks = -1
                If chunk = 0 Then
                    If Not onUploadStart(chunk, chunks, fileName) Then Return
                End If
                If onUploadChunk(fileUpload.InputStream, chunk, chunks, fileName) Then
                    If chunk >= chunks - 1 Then
                        onUploadComplete(fileName)
                    End If ' chunk >= chunks - 1
                    WriteSucessResponse()
                End If ' onUploadChunk(fileUpload.InputStream, chunk, chunks, fileName)
                Return
            End If ' chunks = -1
        End If ' Request.Files.Count > 0
    End Sub

Basically, I check for something on the FILES upload, and check how many chunks are supposed to be sent for this file, and which chunk this particular file item is.  The behavior is slightly different for a file that has a single chunk (if chunks = -1), but the basic steps are:
  1. If the first chunk, do upload start (in this case, create a record in the db indicating a file is upload, and the user its associated with)
  2. Save the uploaded chunk somewhere 
  3. if this is the last chunk, tell the db that all chunks are uploaded

Then performing the actual file saving aspects, went something like this:

    Private Function onUploadChunk(fs As IO.Stream, chunk As Integer, numberOfChunks As Integer, fileName As String) As Boolean
        Dim newFileName As String = ""
        Dim stream As IO.Stream
        Try 
            // set newFileName from DB
            Dim uploadFilePath As String = IO.Path.Combine(Me.fileLocationPath, String.Format("{0}.{1}",newFileName , chunk.ToString("#000")))
            If IO.File.Exists(uploadFilePath) Then IO.File.Delete(uploadFilePath)
            stream = New IO.FileStream(uploadFilePath, IO.FileMode.OpenOrCreate, IO.FileAccess.Write)
            fs.CopyTo(stream, 16384)
        Catch ex As Exception
            ' Need to set status on this file to bad, as  a chunk failed to upload....
            WriteErrorResponse(ex.Message, 100, True)
            Return False
        Finally
            If stream IsNot Nothing Then stream.Dispose()
        End Try
        Return True
    End Function

Again, fairly straightforward code.  Though a few thinks, my logic records the file that this is, into the database, that means that the file that I'm saving to the file-system, only has a GUID for a name. Then the chunks are given an extension of which chunk they actually are. 

Now, I just need to find the best solution for scanning the files as they're uploaded into the network. 



Monday, August 13, 2018

Technical Debt


 Wikipedia defines technical debt as: 
A concept in software development that reflects the implied cost of additional rework caused by choosing an easy solution now instead of using a better approach that would take longer. 
Which is as good of a definition for that as I've seen.  But it doesn't tell the whole story. 

But it's not the definition I think of when I reference it.  And I know that this will get people all sorts of bent out of shape, but when I hear technical debt, I also consider it more along the lines fo this: 

The implied cost of additional work caused by stagnating on the learning of technical concepts, processes, standards or behaviors. 

You can accrue technical debt by choosing the easy solution, instead of the best solution.  But at times, customers can only afford the easy solution now. 

You can accrue technical debt by building project after project using the sames methods and methodologies time after time.  And never looking at newer frameworks and just how other people are doing things.  By not learning things like .NET Core or Node.js because ASP.Net Webforms just work. 

And that's the type of technical debt that I've recently realized that I've been accruing.  I've not learned these new things, because what I've been doing works.  What I'm doing, how I'm building software, is not wrong, it's just I'm blind to the possibility that it's not the best solution for the issue I'm solving.  

Which in turn, drives that initial definition of choosing an easy solution now instead of a better approach that may take longer. 

Blog Widget by LinkWithin