Configuring Nginx & CodeIgniter with Rewrite Rules

I’ve recently begun working with Nginx as we’re finally moving off of Apache in our infrastructure. The first thing I noticed is documentation for various types of configurations is a bit sporadic. A lot is outdated and many don’t use best practices.

The first application I’m working on is a CodeIgniter-based PHP site which needs a few rewrite rules to function properly. I dug up this tutorial but it unfortunately uses a lot of if statements which I quickly learned are evil in Nginx.

Below is the configuration that I’ve settled upon (for now) which takes into account some best practices including:

  • uses php-fpm for the upstream server via a unix socket
  • redirects all http://www.example.com traffic to example.com
  • passes all requests under the root to the front-controller as long as they don’t exist (allows nginx to directly serve static files)
  • prevents uncontrolled requests from being passed to php
  • prevents any access to leftover .htaccess files

I’m hoping to add some of the additional rewrite functionality found in the tutorial at a later date.

upstream php {
server unix:/var/run/php5-fpm.sock;
}
server {
# enforce NO www
server_name http://www.example.com;
return 301 $scheme://example.com$request_uri;
}
server {
listen 80;
server_name example.com;
root /home/webapps/www.example.com/;
access_log /var/log/nginx/www.example.com.access.log main;
location / {
index index.php;
# pass requests to the front controller (http://wiki.nginx.org/Pitfalls#Front_Controller_Pattern_based_packages)
# but don’t proxy everything (http://wiki.nginx.org/Pitfalls#Proxy_Everything)
try_files $uri $uri/ /index.php;
}
location ~ \.php$ {
# dont pass uncontrolled requests to php (http://wiki.nginx.org/Pitfalls#Passing_Uncontrolled_Requests_to_PHP)
try_files $uri =404;
fastcgi_pass php;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
include /etc/nginx/fastcgi_params;
}
# deny access to .htaccess files
location ~ /\.ht {
deny all;
}
}
view rawexample.com.conf hosted with ❤ by GitHub
Advertisements

Lupa Password Root Mysql Linux Centos

What version of mySQL are you using? I”m using 5.7.10 and had the same problem with logging on as root

There is 2 issues – why can’t I log in as root to start with, and why can I not use ‘mysqld_safe` to start mySQL to reset the root password.

I have no answer to setting up the root password during installation, but here’s what you do to reset the root password

Edit the initial root password on install can be found by running

grep 'temporary password' /var/log/mysqld.log

http://dev.mysql.com/doc/refman/5.7/en/linux-installation-yum-repo.html

Cannot insert the value null into column ‘diagram_id’, table dbo.sysdiagrams

Error Condition: Cannot insert the value null into column ‘diagram_id’, table dbo.sysdiagrams

Problem: This sometimes occurs when a SQL Server database is ported from one version to another (such as 2000 to 2005 or 2005 to 2008) or possibly also when using the SSMS for a newer version of SQL Server against and older database.

Fix: Drop and recreated dbo.sysdiagrams making the column diagram_id an identity column.

use [mydatabase]
GO
DROP TABLE [dbo].[sysdiagrams]
GO

CREATE TABLE [dbo].[sysdiagrams]

(   [name] [nvarchar](128) NOT NULL,
[principal_id] [int] NOT NULL,
[diagram_id] [int] identity(1,1) NOT NULL,
[version] [int] NULL,
[definition] [varbinary](max) NULL
)
GO

NOTE: If you want to preserve the diagrams, then rename the table and create a new one, then SELECT – INTO to copy the data into the new table.

RecyclerView – Android

n this post, we are going to load the list of posts from PCSalt.com. In our previous post, JSON Parsing – Android, we used ListView to display a list of posts. But, in this post, we are going to use RecyclerView to display it.

Almost all parsing and model classes are used from that post. If you haven’t followed that post, don’t worry. We are going to write all code here. This project has been uploaded to GitHub.

Create a new Android project using following information.

 

Cannot insert the value null into column ‘diagram_id’, table dbo.sysdiagrams

Error Condition: Cannot insert the value null into column ‘diagram_id’, table dbo.sysdiagrams

Problem: This sometimes occurs when a SQL Server database is ported from one version to another (such as 2000 to 2005 or 2005 to 2008) or possibly also when using the SSMS for a newer version of SQL Server against and older database.

Fix: Drop and recreated dbo.sysdiagrams making the column diagram_id an identity column.

use [mydatabase]
GO
DROP TABLE [dbo].[sysdiagrams]
GO

CREATE TABLE [dbo].[sysdiagrams]

(   [name] [nvarchar](128) NOT NULL,
[principal_id] [int] NOT NULL,
[diagram_id] [int] identity(1,1) NOT NULL,
[version] [int] NULL,
[definition] [varbinary](max) NULL
)
GO

NOTE: If you want to preserve the diagrams, then rename the table and create a new one, then SELECT – INTO to copy the data into the new table.

Netcore API Package

{
“dependencies”: {
“Microsoft.NETCore.App”: {
“version”: “1.0.1”,
“type”: “platform”
},
“Microsoft.ApplicationInsights.AspNetCore”: “1.0.0”,
“Microsoft.AspNetCore.Mvc”: “1.0.1”,
“Microsoft.AspNetCore.Routing”: “1.0.1”,
“Microsoft.AspNetCore.Server.IISIntegration”: “1.0.0”,
“Microsoft.AspNetCore.Server.Kestrel”: “1.0.1”,
“Microsoft.Extensions.Configuration.EnvironmentVariables”: “1.0.0”,
“Microsoft.Extensions.Configuration.FileExtensions”: “1.0.0”,
“Microsoft.Extensions.Configuration.Json”: “1.0.0”,
“Microsoft.Extensions.Logging”: “1.0.0”,
“Microsoft.Extensions.Logging.Console”: “1.0.0”,
“Microsoft.Extensions.Logging.Debug”: “1.0.0”,
“Microsoft.Extensions.Options.ConfigurationExtensions”: “1.0.0”,
“SapientGuardian.EntityFrameworkCore.MySql”: “7.1.9”,
“System.Linq.Dynamic.Core”: “1.0.6.6”,
“Microsoft.Extensions.Caching.Memory”: “1.0.0”,
“Microsoft.AspNetCore.Session”: “1.0.0”,
“Microsoft.AspNetCore.Identity.EntityFrameworkCore”: “1.0.0”,
“Microsoft.EntityFrameworkCore”: “1.0.1”,
“BundlerMinifier.Core”: “2.2.306”,
“Microsoft.IdentityModel.Protocols”: “2.0.0”,
“MySql.Data.Core”: “7.0.4-IR-191”,
“Newtonsoft.Json”: “9.0.1”,
“Microsoft.Extensions.Caching.Abstractions”: “1.0.0”,
“Microsoft.EntityFrameworkCore.InMemory”: “1.1.1”,
“Microsoft.AspNetCore.Authentication.JwtBearer”: “1.0.0”
},

“tools”: {
“Microsoft.AspNetCore.Server.IISIntegration.Tools”: “1.0.0-preview2-final”
},

“frameworks”: {
“netcoreapp1.0”: {
“imports”: [
“dotnet5.6”,
“portable-net45+win8”
]
}
},

“buildOptions”: {
“emitEntryPoint”: true,
“preserveCompilationContext”: true
},

“runtimeOptions”: {
“configProperties”: {
“System.GC.Server”: true
}
},

“publishOptions”: {
“include”: [
“wwwroot”,
“**/*.cshtml”,
“appsettings.json”,
“web.config”
]
},

“scripts”: {
“postpublish”: [ “dotnet publish-iis –publish-folder %publish:OutputPath% –framework %publish:FullTargetFramework%” ]
}
}

Configuring the build environment via gradle.properties

Gradle provides several options that make it easy to configure the Java process that will be used to execute your build. While it’s possible to configure these in your local environment via GRADLE_OPTS or JAVA_OPTS, certain settings like JVM memory settings, Java home, daemon on/off can be more useful if they can be versioned with the project in your VCS so that the entire team can work with a consistent environment. Setting up a consistent environment for your build is as simple as placing these settings into a gradle.properties file. The configuration is applied in following order (if an option is configured in multiple locations the last one wins):

  • from gradle.properties in project build dir.
  • from gradle.properties in gradle user home.
  • from system properties, e.g. when -Dsome.property is set on the command line.

When setting these properties you should keep in mind that Gradle requires a Java JDK or JRE of version 7 or higher to run.

The following properties can be used to configure the Gradle build environment:

org.gradle.daemonWhen set to true the Gradle daemon is used to run the build. For local developer builds this is our favorite property. The developer environment is optimized for speed and feedback so we nearly always run Gradle jobs with the daemon. We don’t run CI builds with the daemon (i.e. a long running process) as the CI environment is optimized for consistency and reliability.

org.gradle.java.homeSpecifies the Java home for the Gradle build process. The value can be set to either a jdk or jre location, however, depending on what your build does, jdk is safer. A reasonable default is used if the setting is unspecified.

org.gradle.jvmargsSpecifies the jvmargs used for the daemon process. The setting is particularly useful for tweaking memory settings. At the moment the default settings are pretty generous with regards to memory.

org.gradle.configureondemandEnables new incubating mode that makes Gradle selective when configuring projects. Only relevant projects are configured which results in faster builds for large multi-projects. See the section called “Configuration on demand”.

org.gradle.parallelWhen configured, Gradle will run in incubating parallel mode.

org.gradle.workers.maxWhen configured, Gradle will use a maximum of the given number of workers. See --max-workers for details.

org.gradle.debugWhen set to true, Gradle will run the build with remote debugging enabled, listening on port 5005. Note that this is the equivalent of adding -agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=5005 to the JVM command line and will suspend the virtual machine until a debugger is attached.

org.gradle.daemon.performance.enable-monitoringWhen set to false, Gradle will not monitor the memory usage of running daemons. See Section 6.5.5, “What can go wrong with Daemon?”.

org.gradle.cachingWhen set to true, Gradle will try to reuse outputs from previous builds. See Section 15.1, “Overview”.

12.1.1. Forked Java processes

Many settings (like the Java version and maximum heap size) can only be specified when launching a new JVM for the build process. This means that Gradle must launch a separate JVM process to execute the build after parsing the various gradle.properties files. When running with the daemon, a JVM with the correct parameters is started once and reused for each daemon build execution. When Gradle is executed without the daemon, then a new JVM must be launched for every build execution, unless the JVM launched by the Gradle start script happens to have the same parameters.

This launching of an extra JVM on every build execution is quite expensive, which is why if you are setting either org.gradle.java.home or org.gradle.jvmargs we highly recommend that you use the Gradle Daemon. See Chapter 6, The Gradle Daemon for more details.

12.2. Gradle properties and system properties

Gradle offers a variety of ways to add properties to your build. With the -D command line option you can pass a system property to the JVM which runs Gradle. The -Doption of the gradle command has the same effect as the -D option of the java command.

You can also add properties to your project objects using properties files. You can place a gradle.properties file in the Gradle user home directory (defined by the “GRADLE_USER_HOME” environment variable, which if not set defaults to USER_HOME/.gradle) or in your project directory. For multi-project builds you can place gradle.properties files in any subproject directory. The properties set in a gradle.properties file can be accessed via the project object. The properties file in the user’s home directory has precedence over property files in the project directories.

You can also add properties directly to your project object via the -P command line option.

Gradle can also set project properties when it sees specially-named system properties or environment variables. This feature is very useful when you don’t have admin rights to a continuous integration server and you need to set property values that should not be easily visible, typically for security reasons. In that situation, you can’t use the -P option, and you can’t change the system-level configuration files. The correct strategy is to change the configuration of your continuous integration build job, adding an environment variable setting that matches an expected pattern. This won’t be visible to normal users on the system. [4]

If the environment variable name looks like ORG_GRADLE_PROJECT_prop=somevalue, then Gradle will set a prop property on your project object, with the value of somevalue. Gradle also supports this for system properties, but with a different naming pattern, which looks like org.gradle.project.prop.

You can also set system properties in the gradle.properties file. If a property name in such a file has the prefix “systemProp.”, like “systemProp.propName”, then the property and its value will be set as a system property, without the prefix. In a multi project build, “systemProp.” properties set in any project except the root will be ignored. That is, only the root project’s gradle.properties file will be checked for properties that begin with the “systemProp.” prefix.

Example 12.1. Setting properties with a gradle.properties file

gradle.properties

gradlePropertiesProp=gradlePropertiesValue
sysProp=shouldBeOverWrittenBySysProp
envProjectProp=shouldBeOverWrittenByEnvProp
systemProp.system=systemValue

build.gradle

task printProps {
    doLast {
        println commandLineProjectProp
        println gradlePropertiesProp
        println systemProjectProp
        println envProjectProp
        println System.properties['system']
    }
}

Output of gradle -q -PcommandLineProjectProp=commandLineProjectPropValue -Dorg.gradle.project.systemProjectProp=systemPropertyValue printProps

> gradle -q -PcommandLineProjectProp=commandLineProjectPropValue -Dorg.gradle.project.systemProjectProp=systemPropertyValue printProps
commandLineProjectPropValue
gradlePropertiesValue
systemPropertyValue
envPropertyValue
systemValue

12.2.1. Checking for project properties

You can access a project property in your build script simply by using its name as you would use a variable. If this property does not exist, an exception will be thrown and the build will fail. If your build script relies on optional properties the user might set, perhaps in a gradle.properties file, you need to check for existence before you access them. You can do this by using the method hasProperty('propertyName') which returns true or false.

12.3. Accessing the web via a proxy

Configuring an HTTP or HTTPS proxy (for downloading dependencies, for example) is done via standard JVM system properties. These properties can be set directly in the build script; for example, setting the HTTP proxy host would be done with System.setProperty('http.proxyHost', 'www.somehost.org'). Alternatively, the properties can be specified in a gradle.properties file, either in the build’s root directory or in the Gradle home directory.

Example 12.2. Configuring an HTTP proxy

gradle.properties

systemProp.http.proxyHost=www.somehost.org
systemProp.http.proxyPort=8080
systemProp.http.proxyUser=userid
systemProp.http.proxyPassword=password
systemProp.http.nonProxyHosts=*.nonproxyrepos.com|localhost

There are separate settings for HTTPS.

Example 12.3. Configuring an HTTPS proxy

gradle.properties

systemProp.https.proxyHost=www.somehost.org
systemProp.https.proxyPort=8080
systemProp.https.proxyUser=userid
systemProp.https.proxyPassword=password
systemProp.https.nonProxyHosts=*.nonproxyrepos.com|localhost

We could not find a good overview for all possible proxy settings. One place to look are the constants in a file from the Ant project. Here’s a link to the repository. The other is a Networking Properties page from the JDK docs. If anyone knows of a better overview, please let us know via the mailing list.

12.3.1. NTLM Authentication

If your proxy requires NTLM authentication, you may need to provide the authentication domain as well as the username and password. There are 2 ways that you can provide the domain for authenticating to a NTLM proxy:

  • Set the http.proxyUser system property to a value like domain/username.
  • Provide the authentication domain via the http.auth.ntlm.domain system property.

Source : https://docs.gradle.org/current/userguide/build_environment.html#sec:accessing_the_web_via_a_proxy

Android – Upload files to ASP.NET Web API service

In previous posts, I showed you how to get/post data to ASP.NET Web API service. However, until now, I only used JSON for communicating between server and client and what about for binary data? For example, uploading files to ASP.NET Web API? How should the controller work and which kind of HTTP request/HTTP format should the Android client send to server? In this blog post, I will show you a simple way to transfer binary data. In the demo, I have 2 components : web service and android client. Unlike the other post, for this one, unfortunately, I don’t have any sample web service (online over internet) for you to test. I don’t own any windows server (neither VPS nor dedicated server :() therefore I can’t host any web service allowing file uploading.

In the download section, at the end of post, you’ll find the source code of web service and Android client. To play the demo, you’ve to publish the service to your localhost and connect your client to it. If you don’t know how to publish a web service to your localhost, please follow instructions here .

Moreover, in this post, I will show you how a web service serves different kinds of client. I have already prepared 4 clients: Android client, console client, drag and drop web client and file browsing web client. The client and server will use HTTP request of mime multipart encoding type for transferring data. You will see this setting applied at all sample clients.

1. Web service

When you open the web service solution, you’ll see there are 3 projects.
– The “Upload File To ASPNET Web API” project is the web service itself and 2 web clients.
– The “Upload File To ASPNET Web API Client” is the console client.
– The “Upload File To ASPNET Web API Models” defines contract between server and client.

In this part, we’ll focus only on the web service to analyze how it works. Go to “Upload File To ASPNET Web API” project –> HDFilesController where the logic code of web service locates.

public Task<IQueryable<HDFile>> Post()
{
    try
    {
        var uploadFolderPath = HostingEnvironment.MapPath("~/App_Data/" + UploadFolder);
        log.Debug(uploadFolderPath);
        //#region CleaningUpPreviousFiles.InDevelopmentOnly
        //DirectoryInfo directoryInfo = new DirectoryInfo(uploadFolderPath);
        //foreach (FileInfo fileInfo in directoryInfo.GetFiles())
        //  fileInfo.Delete();
        //#endregion
        if (Request.Content.IsMimeMultipartContent())
        {
            var streamProvider = new WithExtensionMultipartFormDataStreamProvider(uploadFolderPath);
            var task = Request.Content.ReadAsMultipartAsync(streamProvider).ContinueWith<IQueryable<HDFile>>(t =>
            {
                if (t.IsFaulted || t.IsCanceled)
                {
                    throw new HttpResponseException(HttpStatusCode.InternalServerError);
                }
                var fileInfo = streamProvider.FileData.Select(i =>
                {
                    var info = new FileInfo(i.LocalFileName);
                    return new HDFile(info.Name, Request.RequestUri.AbsoluteUri + "?filename=" + info.Name, (info.Length / 1024).ToString());
                });
                return fileInfo.AsQueryable();
            });
            return task;
        }
        else
        {
            throw new HttpResponseException(Request.CreateResponse(HttpStatusCode.NotAcceptable, "This request is not properly formatted"));
        }
    }
    catch (Exception ex)
    {
        log.Error(ex);
        throw new HttpResponseException(Request.CreateResponse(HttpStatusCode.BadRequest, ex.Message));
    }
}

uploadFolderPath variable determines where the files should be temporarily uploaded into server. I upload directly to App_Data folder but you can set to other folder as you want. Just remember to give full control permission to IUSER so that IIS can write file to that folder.
After setting the folder, the content of request will be validated if it’s a mime-multipart type. If the request is not conformed, web service will reject with HttpStatusCode.NotAcceptable. If the request is correct, the binary data will be extracted from content and IIS stores files in specified location.
However we should always remember that we “never trust user input”. The file name given in HTTP request maybe not conformed (file name in Unix platform can be not accepted in Windows platform), so we have to change the original file name in the request to new file name which we can be sure that it won’t “hurt” our server. I choose the format of {GUID}.{FILE_EXTENSION}, you can choose another format as you like, maybe with time stamp. Just making a class inherits from MultipartFormDataStreamProvider and override the GetLocalFileName with your own logic. For example, my custom MultipartFormDataStreamProvider looks like following

public class WithExtensionMultipartFormDataStreamProvider : MultipartFormDataStreamProvider
{
    public WithExtensionMultipartFormDataStreamProvider(string rootPath)
        : base(rootPath)
    {
    }
    public override string GetLocalFileName(System.Net.Http.Headers.HttpContentHeaders headers)
    {
        string extension = !string.IsNullOrWhiteSpace(headers.ContentDisposition.FileName) ? Path.GetExtension(OSUtil.GetValidFileName(headers.ContentDisposition.FileName)) : "";
        return Guid.NewGuid().ToString() + extension;
    }
}

After all files were successfully uploaded to server, the Post action reply the client with a list of object in JSON format. This list contains many JSON objects with file name (on server), size and URL. Return URL back to client allowing user the possibility to get file back or share file with other user.

I have already extended the web service so that the user can download files not only through the given URLs but also through a GET HTTP request with correct parameter. In the controller, I wrote a GET action receiving file name as parameter, search for file name in server if it exists and give file back.

public HttpResponseMessage Get(string fileName)
{
    HttpResponseMessage result = null;
    DirectoryInfo directoryInfo = new DirectoryInfo(HostingEnvironment.MapPath("~/App_Data/" + UploadFolder));
    FileInfo foundFileInfo = directoryInfo.GetFiles().Where(x => x.Name == fileName).FirstOrDefault();
    if (foundFileInfo != null)
    {
        FileStream fs = new FileStream(foundFileInfo.FullName, FileMode.Open);
        result = new HttpResponseMessage(HttpStatusCode.OK);
        result.Content = new StreamContent(fs);
        result.Content.Headers.ContentType = new System.Net.Http.Headers.MediaTypeHeaderValue("application/octet-stream");
        result.Content.Headers.ContentDisposition = new ContentDispositionHeaderValue("attachment");
        result.Content.Headers.ContentDisposition.FileName = foundFileInfo.Name;
    }
    else
    {
        result = new HttpResponseMessage(HttpStatusCode.NotFound);
    }
    return result;
}

That is all about our web service. One POST action for posting binary data from client to server and one GET action for give binary data back from server to client. That means we have 2 ways binary data communicating channel.

2. Clients

In this second part, we build up our clients. As I mentioned at the beginning, I will introduce 4 types of client to prove that we can use our web service for many client system. Important: Remember to start you web service when you connect the clients to it. :).

2.1 Android client

Before getting started, I recommend you to read this article first (if you didn’t) . I will use dependency injection in example, it’s nothing special. Just if you don’t know how DI works, it’s very difficult for you to understand how the code works later.

The second note is Android client can only access the web service at your real localhost. He can’t access the web service in Debug mode of Visual Studio (for example http://localhost:4260/…). So, you have to really publish the web service code to your localhost. If you don’t know how to publish the web service, you can read the instructions at the link I post at the beginning.

In Android client, there are only 2 activities: the main activity and the result activity. The main activity is a list activity where I will show all files under “sdcard/wallpapers” folder. When the Android client starts, I will list all files in folder, pop them up the main activity so that user can choose which file they would like to upload to server.

public List<HDFile> getLocalFiles() {
    List<HDFile> result = new ArrayList<HDFile>();
    String sdCard = Environment.getExternalStorageDirectory().getAbsolutePath();
    String wallpaperPath = sdCard + "/wallpapers/";
    File files[] = new File(wallpaperPath).listFiles();
    for(int index=0;index < files.length;index++)
    {
        HDFile wallpaper = new HDFile();
        wallpaper.setId(CryptoUtil.Md5(files[index].getName()));
        wallpaper.setName(files[index].getName());
        wallpaper.setFilePath(IOUtil.pathCombine(wallpaperPath,files[index].getName()));
        result.add(wallpaper);
    }
    return result;
}

The code listing above will get path of “sdcard”, find the folder “wallpapers” beneath it and then push all files into a list. HDFile is the contract object for communicating between server and client. CryptoUtil and IOUtil are just helper utilization class. You can check the code out from the repositories at the end of post.

After the local files were loaded successfully, I will display them on our main activity with check boxes so that we can tick which files we want to upload. However, as default, the list activity doesn’t provide us a layout with check box in front. Therefore, we need to define an adapter to customize the layout as following

public class ListViewLocalFileAdapter extends ArrayAdapter<HDFile> {
    private List<HDFile> hdFiles;
    private Context context;
    public ListViewLocalFileAdapter(Context context, int textViewResourceId, List<HDFile> objects) {
        super(context, textViewResourceId, objects);
        this.context = context;
        this.hdFiles = objects;
    }
    static class ViewHolder
    {
        protected CheckBox checkBox;
        protected TextView textViewName;
    }
    @Override
    public View getView(int position, View convertView, ViewGroup parent) {
        View view = null;
        if (convertView == null)
        {
            LayoutInflater inflater = (LayoutInflater) context.getSystemService(Context.LAYOUT_INFLATER_SERVICE);
            view = inflater.inflate(R.layout.list_file_item,null);
            final ViewHolder viewHolder = new ViewHolder();
            viewHolder.checkBox = (CheckBox)view.findViewById(R.id.checkBoxSelected);
            viewHolder.checkBox.setOnCheckedChangeListener(CheckBoxOnCheckedChangeListener);
            viewHolder.textViewName= (TextView)view.findViewById(R.id.textViewName);
            view.setTag(viewHolder);
            viewHolder.checkBox.setTag(hdFiles.get(position));
        }
        else
        {
            view = convertView;
            ((ViewHolder)view.getTag()).checkBox.setTag(hdFiles.get(position));
        }
        ViewHolder viewHolder=(ViewHolder)view.getTag();
        viewHolder.textViewName.setText(hdFiles.get(position).getName());
        viewHolder.checkBox.setChecked(hdFiles.get(position).isSelected());
        return view;
    }
    private CompoundButton.OnCheckedChangeListener CheckBoxOnCheckedChangeListener = new CompoundButton.OnCheckedChangeListener() {
        @Override
        public void onCheckedChanged(CompoundButton buttonView, boolean isChecked) {
            CheckBox checkBox = (CheckBox)buttonView;
            HDFile hdFile = (HDFile) checkBox.getTag();
            hdFile.setSelected(isChecked);
        }
    };
    public List<HDFile> getItems()
    {
        return hdFiles;
    }
}

And then when the app starts, the main activity will look like this

List With Checkbox activity

Remember that the sample files are loaded from “sdcard/wallpapers” folder, be sure that you have some sample files in that folder and of course, the folder “wallpapers” must also exist.

Android Debug Monitor File Explorer

To upload file to server, just choose some files and tap on the upload button on the action bar. That will make a HTTP POST to server and upload our files.

<?xml version="1.0" encoding="utf-8"?>
    <item android:id="@+id/main_menu_upload"
          android:icon="@drawable/ic_action_upload"
          android:title="Add"
          android:showAsAction="ifRoom|withText" />
</menu>
@Override
public boolean onCreateOptionsMenu(Menu menu) {
    MenuInflater menuInflater = getMenuInflater();
    menuInflater.inflate(R.menu.menumain, menu);
    return true;
}
public boolean onOptionsItemSelected(MenuItem item) {
    switch (item.getItemId()) {
        case R.id.main_menu_upload:
            List<HDFile> selectedFiles = new ArrayList<HDFile>();
            for(HDFile file:adapter.getItems())
            {
                if (file.isSelected())
                {
                    selectedFiles.add(file);
                }
            }
            if (selectedFiles.size()>0)
                viewModel.uploadFiles(selectedFiles);
            else
                AlertMessageBox.Show(getApplicationContext(),"Info","Please select a file to upload", AlertMessageBox.AlertMessageBoxIcon.Info);
            return true;
        default:
            return super.onOptionsItemSelected(item);
    }
}
...
public void uploadFiles(List<HDFile> files) {
    new UploadFilesTask().execute(files.toArray(new HDFile[files.size()]));
}
class UploadFilesTask extends AsyncTask<HDFile,String,Integer>
{
    Integer totalCount = 0;
    @Override
    protected Integer doInBackground(HDFile... params) {
        Integer uploadCount =0;
        totalCount = params.length;
        uploadedFiles = new ArrayList<HDFile>();
        for(int index=0;index < params.length;index++)
        {
            File file = new File(params[index].getFilePath());
            JSONHttpClient jsonHttpClient = new JSONHttpClient();
            HDFile[] hdFiles = jsonHttpClient.PostFile(ServiceUrl.REST_SERVICE_URL, params[index].getId(),file,params[index].getName(),HDFile[].class);
            if (hdFiles != null && hdFiles.length == 1)
            {
                uploadCount++;
                uploadedFiles.add(hdFiles[0]);
            }
        }
        return uploadCount;
    }
    @Override
    protected void onPostExecute(Integer uploadCount) {
        UploadErrorCode errorCode = UploadErrorCode.OK;
        if (uploadCount ==0)
            errorCode = UploadErrorCode.Failed;
        else if (uploadCount < totalCount)
            errorCode = UploadErrorCode.PartlySuccessful;
        if (uploadFilesCompleteListener != null)
            uploadFilesCompleteListener.onCompleted(errorCode);
    }
}

The files will be uploaded one by one to server. You can extend my code for notifying upload progress changes. According to the count of uploaded files, we send appropriate error code back (failed, partly successful or OK). When upload progress finishes, the information of uploaded files will be shown on the result activity.

Uploaded files

The code for this activity is pretty simple, I think you can read and understand yourself.

2.2 .Net client

We have checked how the Android client works. In this part, I will show you how to consume our web service in a desktop application. The .net client is much more simple than Android client, there is no UI for it. 🙂 . I just make a simple console application, create a HTTP post with MultipartFormDataContent, push the content to it and send it to server.

private static async void UploadFiles()
{
    Uri server = new Uri("http://localhost:4260/api/hdfiles");
    HttpClient httpClient = new HttpClient();
    StringContent stringContent = new StringContent("Broken Sword: The Shadow of the Templars (also known as Circle of Blood in the United States)[1] is a 1996 point-and-click adventure game developed by Revolution Software. The player assumes the role of George Stobbart, an American tourist in Paris, as he attempts to unravel a conspiracy. The game takes place in both real and fictional locations in Europe and the Middle East.", Encoding.UTF8, "text/plain");
    StreamContent streamConent = new StreamContent(new FileStream(@"..\..\TestData\HintDesk.png", FileMode.Open, FileAccess.Read, FileShare.Read));
    MultipartFormDataContent multipartFormDataContent = new MultipartFormDataContent();
    multipartFormDataContent.Add(stringContent, "Broken Sword", "Broken Sword.txt");
    multipartFormDataContent.Add(streamConent, "HintDesk", "HintDesk.png");
    //HttpResponseMessage responseMessage = await httpClient.PostAsync(server, multipartFormDataContent);
    HttpResponseMessage responseMessage = httpClient.PostAsync(server, multipartFormDataContent).Result;
    if (responseMessage.IsSuccessStatusCode)
    {
        IList<HDFile> hdFiles = await responseMessage.Content.ReadAsAsync<IList<HDFile>>();
        if (Directory.Exists(DownloadFolder))
            (new DirectoryInfo(DownloadFolder)).Empty();
        else
            Directory.CreateDirectory(DownloadFolder);
        foreach (HDFile hdFile in hdFiles)
        {
            responseMessage = httpClient.GetAsync(new Uri(hdFile.Url)).Result;
            if (responseMessage.IsSuccessStatusCode)
            {
                using (FileStream fs = File.Create(Path.Combine(DownloadFolder, hdFile.Name)))
                {
                    Stream streamFromService = await responseMessage.Content.ReadAsStreamAsync();
                    streamFromService.CopyTo(fs);
                }
            }
        }
    }
}

The HTTP request of code listing above contains 2 files “Broken Sword.txt”, and “HintDesk.png”. After posting files successfully, I will download them back again, just for testing if everything works. You see, the .net client is not much different from unit test but it’s good example to show how to consume the web service in desktop application.

2.3 Web client: Drag and Drop

In this part, we’ll discuss our 3rd client: the web client with drag and drop. We’ll use feature “drag and drop” of HTML5 to allow user to drag/drop file from his local computer to server.

Web client drag and drop

The source code of this client is in the same project of web service. Browse to Views –> DirectUpload –> Index.cshtml is where the layout of this client locates.

Drag And Drop HTML

In the Index.cshtml is the HTML code for building up the container where user can drop files and another container for showing the information of uploaded files. The javascript code for handling drag and drop is in the “~/bundles/draganddropscripts” defined in BundleConfig.cs

bundles.Add(new ScriptBundle("~/bundles/draganddropscripts")
    .Include("~/Scripts/knockout-2.2.0.js")
    .Include("~/Scripts/fileuploadbydraganddrop.js")
    );

In fileuploadbydraganddrop.js is where we will handle the drop event

function drop(evt) {
    evt.stopPropagation();
    evt.preventDefault();
    $(evt.target).removeClass('over');
    var files = evt.originalEvent.dataTransfer.files;
    if (files.length > 0) {
        if (window.FormData !== undefined) {
            var data = new FormData();
            for (i = 0; i < files.length; i++) {
                data.append("file" + i, files[i]);
            }
            $("#uploadNotification").show();
            $.ajax({
                type: "POST",
                url: root + "api/hdfiles",
                contentType: false,
                processData: false,
                data: data,
                success: function (res) {
                    $.each(res, function (i, item) {
                        viewModel.uploads.push(item);
                    });
                    $("#uploadNotification").hide();
                }
            });
        } else {
            alert("Your browser needs to support HTML5.");
        }
    }
}

After files are dropped to container, an ajax call will be sent to HDFilesController. When ajax call finishes, result will pushed back to viewModel. Then Knockout.js will bind data to result container.

2.4 Web client: Form

Our last client is also a web client. However, we use now the web client using form object. This web client works as same as web client with drag and drop. However I have to combine HTML and Javascript to display the uploading progress like following

<section class="content-wrapper main-content clear-fix">
    <div id="divFormUpload">
        <form id="formUpload" action="@Url.Content("~/api/hdfiles")" method="post" enctype="multipart/form-data">
            <label class="displayinline">Files</label>
            <input name="inputFiles" type="file" multiple />
            <input type="submit" value="Upload" />
            <div id="divProgress" style="display: none">
                <div id="divPercentBar" class="classDivPercentBar"></div>
                <div id="divPercentText" class="classDivPercentText">0%</div>
                @*<img alt="" id="upload-animation" src="/Content/images/loading.gif" />*@
            </div>
        </form>
    </div>
    <ul data-bind="template: { name: 'fileContainer', foreach: uploads }"></ul>
    <script type="text/html" id="fileContainer">
        <li>
            <span>Uploaded: <small data-bind="text: Name"></small>.</span>
            <span>Size: <small data-bind="text: Size"></small>kB</span>
            <span>Server path: <small><a data-bind="    attr: { href: Url }, text: Url"></a></small></span>
        </li>
    </script>
</section>

 

$(document).ready(function () {
    var divProgress = $("#divProgress");
    var divBar = $("#divPercentBar");
    var divPercent = $("#divPercentText");
    $("#formUpload").ajaxForm({
        beforeSend: function () {
            divProgress.show();
            var percentVal = '0%';
            divBar.width(percentVal)
            divPercent.html(percentVal);
        },
        uploadProgress: function (event, position, total, percentComplete) {
            var percentVal = percentComplete + '%';
            divBar.width(percentVal)
            divPercent.html(percentVal);
        },
        success: function (res) {
            var percentVal = '100%';
            divBar.width(percentVal)
            divPercent.html(percentVal);
            $.each(res, function (i, item) {
                viewModel.uploads.push(item);
            });
        },
        complete: function (xhr) {
            //divProgress.hide();
        }
    });
});
var viewModel = {
    uploads: ko.observableArray([])
}
ko.applyBindings(viewModel);

Web client form

The api controller URL isn’t set in javascript code but directly in HTML. The javascript is just for displaying the progress changes.

3. Conclusion

– Now we’re at the end of the post. We’ve already learnt how to transfer binary data between server and client. We already know how to consume the web service with many types of client.
– The source code of server and other client can be downloaded from following link: https://bitbucket.org/hintdesk/dotnet-upload-files-to-asp.net-web-api-service
– Android client can be downloaded from following link https://bitbucket.org/hintdesk/android-upload-files-to-asp.net-web-api-service
– Util classes can be checked out from https://bitbucket.org/hintdesk/android-hintdesk-core

4. Updates

4.1. Update 31.01.2014

The standard limitation for file size in IIS is about 4MB. If you want to allow uploading bigger file, you can add following setting in Web.config, for example, this config below allows file size up to 100MB.

<system.web>
...
<httpRuntime targetFramework="4.5" maxRequestLength="102400" />
...
</system.web>
<system.webServer>
    <security>
      <requestFiltering>
        <requestLimits maxAllowedContentLength="104857600" />
      </requestFiltering>
    </security>
</system.webServer>

Please note that maxRequestLength is measured in Kb and maxAllowedContentLength in bytes. Both values above equal to 100Mbytes.

4.2. Update 18.04.2014

In the demo Android client when a file is posted to web service its name part will be set by Id

HDFile[] hdFiles = jsonHttpClient.PostFile(ServiceUrl.REST_SERVICE_URL, params[index].getId(), file, params[index].getName(), HDFile[].class);

This name part you can read it again by accessing FileData.Headers.ContentDisposition.Parameters

Get Id from Post request